[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:29:23 dist-test-slave-j2vl krb5kdc[30826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:29:23 dist-test-slave-j2vl krb5kdc[30826](info): set up 2 sockets
Aug 09 16:29:23 dist-test-slave-j2vl krb5kdc[30826](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 16:29:25.091414 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.530s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:29:25 dist-test-slave-j2vl krb5kdc[30826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:29:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:29:25.238175 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:36337
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:43241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:36337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:25.329211 30844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:25.329535 30844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:25.329582 30844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:25.331046 30844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:25.331154 30844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:25.331185 30844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:25.331219 30844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:25.331247 30844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:25.332955 30844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:36337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:36337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:25.333295 30844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:25.333596 30844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:25.339417 30857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:25.339593 30852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:25.339437 30844 server_base.cc:1034] running on GCE node
W20260809 16:29:25.339418 30853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:25.339988 30844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:25.340309 30844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:25.341459 30844 hybrid_clock.cc:648] HybridClock initialized: now 1786292965341439 us; error 33 us; skew 500 ppm
Aug 09 16:29:25 dist-test-slave-j2vl krb5kdc[30826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292965, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:25.344502 30844 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:29:25.345052 30844 webserver.cc:460] Webserver started at http://127.30.23.190:42221/ using document root <none> and password file <none>
I20260809 16:29:25.345310 30844 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:25.345407 30844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:25.345564 30844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:25.346743 30844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "bbca1c1a73db43bfaf12507b2b613faf"
format_stamp: "Formatted at 2026-08-09 16:29:25 on dist-test-slave-j2vl"
server_key: "6daf4ee425cdcd72f265064fcd9cbead"
server_key_iv: "0670b163177f72f670926d87a6ef8da5"
server_key_version: "encryptionkey@0"
I20260809 16:29:25.347193 30844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "bbca1c1a73db43bfaf12507b2b613faf"
format_stamp: "Formatted at 2026-08-09 16:29:25 on dist-test-slave-j2vl"
server_key: "6daf4ee425cdcd72f265064fcd9cbead"
server_key_iv: "0670b163177f72f670926d87a6ef8da5"
server_key_version: "encryptionkey@0"
I20260809 16:29:25.348764 30844 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:29:25.349718 30868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:25.349963 30844 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:25.350095 30844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "bbca1c1a73db43bfaf12507b2b613faf"
format_stamp: "Formatted at 2026-08-09 16:29:25 on dist-test-slave-j2vl"
server_key: "6daf4ee425cdcd72f265064fcd9cbead"
server_key_iv: "0670b163177f72f670926d87a6ef8da5"
server_key_version: "encryptionkey@0"
I20260809 16:29:25.350180 30844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:25.366688 30844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:25.369626 30844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:25.369837 30844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:25.374446 30844 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:36337
I20260809 16:29:25.374497 30954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:36337 every 8 connection(s)
I20260809 16:29:25.375002 30844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:25.376022 30955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:25.378623 30955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf: Bootstrap starting.
I20260809 16:29:25.379283 30955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:25.379604 30955 log.cc:824] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:25.380479 30955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf: No bootstrap required, opened a new log
I20260809 16:29:25.382309 30955 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbca1c1a73db43bfaf12507b2b613faf" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36337 } }
I20260809 16:29:25.382460 30955 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:25.382485 30955 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbca1c1a73db43bfaf12507b2b613faf, State: Initialized, Role: FOLLOWER
I20260809 16:29:25.382669 30955 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbca1c1a73db43bfaf12507b2b613faf" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36337 } }
I20260809 16:29:25.382761 30955 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:25.382787 30955 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:25.382819 30955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:25.383460 30955 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbca1c1a73db43bfaf12507b2b613faf" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36337 } }
I20260809 16:29:25.383560 30955 leader_election.cc:302] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbca1c1a73db43bfaf12507b2b613faf; no voters:
I20260809 16:29:25.383785 30955 leader_election.cc:288] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:25.383864 30960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:25.384166 30960 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [term 1 LEADER]: Becoming Leader. State: Replica: bbca1c1a73db43bfaf12507b2b613faf, State: Running, Role: LEADER
I20260809 16:29:25.384325 30955 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:25.384354 30960 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbca1c1a73db43bfaf12507b2b613faf" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36337 } }
I20260809 16:29:25.385388 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 30844
I20260809 16:29:25.385525 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:25.385473 30963 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbca1c1a73db43bfaf12507b2b613faf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbca1c1a73db43bfaf12507b2b613faf" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36337 } } }
I20260809 16:29:25.385582 30963 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:25.385927 30976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:25.385481 30966 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbca1c1a73db43bfaf12507b2b613faf. Latest consensus state: current_term: 1 leader_uuid: "bbca1c1a73db43bfaf12507b2b613faf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbca1c1a73db43bfaf12507b2b613faf" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36337 } } }
I20260809 16:29:25.385990 30966 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:25.386397 30814 external_mini_cluster.cc:1428] Setting key 478564ce0fe7e758d84f2c65e7b69487
I20260809 16:29:25.386600 30976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:25.388712 30976 catalog_manager.cc:1347] Generated new cluster ID: b851f4882c21442d953596be83af57a4
I20260809 16:29:25.388777 30976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:29:25 dist-test-slave-j2vl krb5kdc[30826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:25.408993 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:25.387511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56504 (local address 127.30.23.190:36337)
0809 16:29:25.387719 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:25.387728 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:25.388132 (+ 404us) server_negotiation.cc:407] Connection header received
0809 16:29:25.390733 (+ 2601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:25.390740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:25.390813 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:25.390951 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:25.393187 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:25.395187 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:25.396807 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:25.397460 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:25.397567 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:25.402716 (+ 5149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:25.402722 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:25.402723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:25.402738 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:25.405377 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:25.406244 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:25.406249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:25.406250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:25.406400 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:25.407009 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:25.407010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:25.407010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:25.407384 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:25.407624 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:25.408276 (+ 652us) server_negotiation.cc:299] Negotiation successful
0809 16:29:25.408451 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260809 16:29:25.409675 30976 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:25.410323 30976 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:25.425043 30976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbca1c1a73db43bfaf12507b2b613faf: Generated new TSK 0
I20260809 16:29:25.425290 30976 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:29:25.465917 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:36337
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:43241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:25.599858 30997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:25.600140 30997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:25.600172 30997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:25.602644 30997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:25.602726 30997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:25.602763 30997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:29:25.605762 30997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:36337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:25.606060 30997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:25.606530 30997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:25.614114 31008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:25.614114 31010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:25.614172 31012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:25.614550 30997 server_base.cc:1034] running on GCE node
I20260809 16:29:25.614814 30997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:25.615067 30997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:25.616247 30997 hybrid_clock.cc:648] HybridClock initialized: now 1786292965616218 us; error 59 us; skew 500 ppm
Aug 09 16:29:25 dist-test-slave-j2vl krb5kdc[30826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292965, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:25.620860 30997 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:29:25.621510 30997 webserver.cc:460] Webserver started at http://127.30.23.129:36901/ using document root <none> and password file <none>
I20260809 16:29:25.621783 30997 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:25.621881 30997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:25.622054 30997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:25.623286 30997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "26ed4184c9d6442b96e4917743307c36"
format_stamp: "Formatted at 2026-08-09 16:29:25 on dist-test-slave-j2vl"
server_key: "b5d2175b694df93909a7135e082d466c"
server_key_iv: "ffe7e02eea43067ed53d879d17f4dbce"
server_key_version: "encryptionkey@0"
I20260809 16:29:25.623739 30997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "26ed4184c9d6442b96e4917743307c36"
format_stamp: "Formatted at 2026-08-09 16:29:25 on dist-test-slave-j2vl"
server_key: "b5d2175b694df93909a7135e082d466c"
server_key_iv: "ffe7e02eea43067ed53d879d17f4dbce"
server_key_version: "encryptionkey@0"
I20260809 16:29:25.625777 30997 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 16:29:25.627306 31022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:25.627771 30997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:29:25.627956 30997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "26ed4184c9d6442b96e4917743307c36"
format_stamp: "Formatted at 2026-08-09 16:29:25 on dist-test-slave-j2vl"
server_key: "b5d2175b694df93909a7135e082d466c"
server_key_iv: "ffe7e02eea43067ed53d879d17f4dbce"
server_key_version: "encryptionkey@0"
I20260809 16:29:25.628101 30997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:25.644608 30997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:25.648442 30997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:25.648650 30997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:25.649025 30997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:25.649518 30997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:25.649585 30997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:25.649629 30997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:25.649657 30997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:25.667230 30997 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:34203
I20260809 16:29:25.667321 31215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:34203 every 8 connection(s)
I20260809 16:29:25.667709 30997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:29:25.674549 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 30997
I20260809 16:29:25.674674 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:29:25.674856 30814 external_mini_cluster.cc:1428] Setting key 9ff83d714367d313238d397422076c46
Aug 09 16:29:25 dist-test-slave-j2vl krb5kdc[30826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292965, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:25.682685 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:25.668985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:46829 (local address 127.30.23.190:36337)
0809 16:29:25.669086 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:25.669088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:25.669222 (+ 134us) server_negotiation.cc:407] Connection header received
0809 16:29:25.670219 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:25.670221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:25.670237 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:25.670305 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:25.671678 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:25.672202 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:25.673569 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:25.673785 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:25.673861 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:25.678002 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:25.678004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:25.678004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:25.678013 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:25.680226 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:25.680812 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:25.680814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:25.680814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:25.680897 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:25.681565 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:25.681567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:25.681568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:25.681751 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:25.681872 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:25.682446 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 16:29:25.682567 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 16:29:25.683344 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:25.668743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:36337 (local address 127.30.23.129:46829)
0809 16:29:25.669027 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:25.669076 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:25.670060 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:25.670461 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:25.670466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:25.671211 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:25.671543 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:25.671548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:25.672323 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:25.672325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:25.673426 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:25.673430 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:25.673908 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:25.673909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:25.674128 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:25.674939 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:25.674953 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:25.677796 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:25.680410 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:25.680420 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:25.680423 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:25.680705 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:25.681050 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:25.681052 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:25.681053 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:25.681460 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:25.681869 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:25.681890 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:25.682120 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 16:29:25.682496 (+ 376us) client_negotiation.cc:240] Negotiation successful
0809 16:29:25.682718 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":161,"threads_started":1}
I20260809 16:29:25.684705 31217 heartbeater.cc:347] Connected to a master server at 127.30.23.190:36337
I20260809 16:29:25.684867 31217 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:25.685117 31217 heartbeater.cc:511] Master 127.30.23.190:36337 requested a full tablet report, sending...
I20260809 16:29:25.686086 30902 ts_manager.cc:194] Registered new tserver with Master: 26ed4184c9d6442b96e4917743307c36 (127.30.23.129:34203)
I20260809 16:29:25.687251 30902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:46829
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:29:25.717818 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:36337
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:43241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:25.818831 31224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:25.819140 31224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:25.819188 31224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:25.820647 31224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:25.820739 31224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:25.820791 31224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:29:25.822674 31224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:36337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:25.822997 31224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:25.823266 31224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:25.830341 31234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:25.830467 31232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:25.830387 31230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:25.830801 31224 server_base.cc:1034] running on GCE node
I20260809 16:29:25.831080 31224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:25.831442 31224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:25.832634 31224 hybrid_clock.cc:648] HybridClock initialized: now 1786292965832623 us; error 31 us; skew 500 ppm
Aug 09 16:29:25 dist-test-slave-j2vl krb5kdc[30826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292965, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:25.835559 31224 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:29:25.836078 31224 webserver.cc:460] Webserver started at http://127.30.23.130:42289/ using document root <none> and password file <none>
I20260809 16:29:25.836300 31224 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:25.836344 31224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:25.836443 31224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:25.837591 31224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "16e53b6f61884f6ba362bb98faeb389d"
format_stamp: "Formatted at 2026-08-09 16:29:25 on dist-test-slave-j2vl"
server_key: "65b11da0408c047d3cc6218ad9dcb7b1"
server_key_iv: "7bb3b421866cea1e044e12493a4f54b3"
server_key_version: "encryptionkey@0"
I20260809 16:29:25.837929 31224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "16e53b6f61884f6ba362bb98faeb389d"
format_stamp: "Formatted at 2026-08-09 16:29:25 on dist-test-slave-j2vl"
server_key: "65b11da0408c047d3cc6218ad9dcb7b1"
server_key_iv: "7bb3b421866cea1e044e12493a4f54b3"
server_key_version: "encryptionkey@0"
I20260809 16:29:25.839399 31224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:25.840224 31244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:25.840473 31224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:25.840557 31224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "16e53b6f61884f6ba362bb98faeb389d"
format_stamp: "Formatted at 2026-08-09 16:29:25 on dist-test-slave-j2vl"
server_key: "65b11da0408c047d3cc6218ad9dcb7b1"
server_key_iv: "7bb3b421866cea1e044e12493a4f54b3"
server_key_version: "encryptionkey@0"
I20260809 16:29:25.840608 31224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:25.871944 31224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:25.874536 31224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:25.874667 31224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:25.874907 31224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:25.875289 31224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:25.875330 31224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:25.875361 31224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:25.875375 31224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:25.882632 31224 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:35919
I20260809 16:29:25.882700 31435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:35919 every 8 connection(s)
I20260809 16:29:25.883078 31224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:29:25.885259 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 31224
I20260809 16:29:25.885419 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:29:25.885660 30814 external_mini_cluster.cc:1428] Setting key 4f9b378a6aa62e5716ec0ba0f3f69d9b
Aug 09 16:29:25 dist-test-slave-j2vl krb5kdc[30826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292965, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:25.895798 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:25.884178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:47229 (local address 127.30.23.190:36337)
0809 16:29:25.884257 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:25.884260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:25.884397 (+ 137us) server_negotiation.cc:407] Connection header received
0809 16:29:25.884933 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:25.884934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:25.884946 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:25.885018 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:25.886369 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:25.887059 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:25.888300 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:25.888504 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:25.888566 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:25.891527 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:25.891529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:25.891530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:25.891539 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:25.893500 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:25.894337 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:25.894339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:25.894339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:25.894435 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:25.894854 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:25.894855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:25.894856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:25.895014 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:25.895207 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:25.895556 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 16:29:25.895661 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 16:29:25.896044 31441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:25.884001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:36337 (local address 127.30.23.130:47229)
0809 16:29:25.884183 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:25.884229 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:25.884779 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:25.885107 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:25.885111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:25.885965 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:25.886229 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:25.886234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:25.887168 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:25.887169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:25.888188 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:25.888193 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:25.888597 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:25.888598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:25.888768 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:25.889451 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:25.889464 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:25.891406 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:25.893615 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:25.893626 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:25.893629 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:25.894210 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:25.894623 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:25.894624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:25.894625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:25.894771 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:25.895193 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:25.895203 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:25.895396 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 16:29:25.895531 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 16:29:25.895732 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
I20260809 16:29:25.896677 31436 heartbeater.cc:347] Connected to a master server at 127.30.23.190:36337
I20260809 16:29:25.896793 31436 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:25.897027 31436 heartbeater.cc:511] Master 127.30.23.190:36337 requested a full tablet report, sending...
I20260809 16:29:25.897622 30902 ts_manager.cc:194] Registered new tserver with Master: 16e53b6f61884f6ba362bb98faeb389d (127.30.23.130:35919)
I20260809 16:29:25.897994 30902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:47229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:29:25.921077 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:36337
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:43241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:26.011193 31446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:26.011410 31446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:26.011436 31446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:26.012610 31446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:26.012656 31446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:26.012686 31446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:29:26.014394 31446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:36337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:26.014679 31446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:26.014947 31446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:26.020390 31462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:26.020416 31458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:26.020435 31456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:26.020716 31446 server_base.cc:1034] running on GCE node
I20260809 16:29:26.020951 31446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:26.021206 31446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:26.022397 31446 hybrid_clock.cc:648] HybridClock initialized: now 1786292966022391 us; error 52 us; skew 500 ppm
Aug 09 16:29:26 dist-test-slave-j2vl krb5kdc[30826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292966, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:26.025156 31446 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:29:26.025696 31446 webserver.cc:460] Webserver started at http://127.30.23.131:37317/ using document root <none> and password file <none>
I20260809 16:29:26.025959 31446 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:26.026031 31446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:26.026180 31446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:26.027247 31446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "87bb0dd6085946bd98d7ddc8340545f8"
format_stamp: "Formatted at 2026-08-09 16:29:26 on dist-test-slave-j2vl"
server_key: "cca397d030d1021cce862460448709d7"
server_key_iv: "845fe724fe0d0c710e9f818fd356ea6c"
server_key_version: "encryptionkey@0"
I20260809 16:29:26.027652 31446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "87bb0dd6085946bd98d7ddc8340545f8"
format_stamp: "Formatted at 2026-08-09 16:29:26 on dist-test-slave-j2vl"
server_key: "cca397d030d1021cce862460448709d7"
server_key_iv: "845fe724fe0d0c710e9f818fd356ea6c"
server_key_version: "encryptionkey@0"
I20260809 16:29:26.029150 31446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:29:26.030082 31475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:26.030294 31446 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:26.030407 31446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "87bb0dd6085946bd98d7ddc8340545f8"
format_stamp: "Formatted at 2026-08-09 16:29:26 on dist-test-slave-j2vl"
server_key: "cca397d030d1021cce862460448709d7"
server_key_iv: "845fe724fe0d0c710e9f818fd356ea6c"
server_key_version: "encryptionkey@0"
I20260809 16:29:26.030494 31446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:26.040241 31446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:26.042588 31446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:26.042768 31446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:26.043038 31446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:26.043370 31446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:26.043423 31446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:26.043466 31446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:26.043495 31446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:26.051517 31446 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:41283
I20260809 16:29:26.051566 31652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:41283 every 8 connection(s)
I20260809 16:29:26.051971 31446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:29:26.058002 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 31446
I20260809 16:29:26.058142 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:29:26.058363 30814 external_mini_cluster.cc:1428] Setting key e689bdfa1afb2836e4ac0e4a6ead23fd
Aug 09 16:29:26 dist-test-slave-j2vl krb5kdc[30826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292966, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:26.064085 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:26.052907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:36337 (local address 127.30.23.131:50495)
0809 16:29:26.053114 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:26.053155 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:26.053766 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:26.054087 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:26.054090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:26.054756 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:26.054969 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:26.054974 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:26.055771 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:26.055773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:26.056703 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:26.056706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:26.057564 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:26.057565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:26.057698 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:26.058517 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:26.058527 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:26.060500 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:26.062426 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:26.062433 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:26.062436 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:26.062705 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:26.063003 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:26.063004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:26.063005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:26.063122 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:26.063489 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:26.063495 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:26.063635 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 16:29:26.063738 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 16:29:26.063906 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":89,"threads_started":1}
I20260809 16:29:26.064116 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:26.052894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:50495 (local address 127.30.23.190:36337)
0809 16:29:26.052946 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:26.052949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:26.053270 (+ 321us) server_negotiation.cc:407] Connection header received
0809 16:29:26.053909 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:26.053910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:26.053932 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:26.053990 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:26.055146 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:26.055660 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:26.056869 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:26.057459 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:26.057529 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:26.060620 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:26.060621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:26.060622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:26.060630 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:26.062322 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:26.062826 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:26.062827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:26.062827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:26.062905 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:26.063218 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:26.063219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:26.063219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:26.063389 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:26.063457 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:26.063793 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 16:29:26.063873 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:29:26.064848 31653 heartbeater.cc:347] Connected to a master server at 127.30.23.190:36337
I20260809 16:29:26.064989 31653 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:26.065212 31653 heartbeater.cc:511] Master 127.30.23.190:36337 requested a full tablet report, sending...
I20260809 16:29:26.065672 30902 ts_manager.cc:194] Registered new tserver with Master: 87bb0dd6085946bd98d7ddc8340545f8 (127.30.23.131:41283)
I20260809 16:29:26.066088 30902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:50495
I20260809 16:29:26.071658 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:29:26 dist-test-slave-j2vl krb5kdc[30826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292966, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:29:26 dist-test-slave-j2vl krb5kdc[30826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292966, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.23.129@KRBTEST.COM
I20260809 16:29:26.100062 31666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:26.090558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34374 (local address 127.30.23.129:34203)
0809 16:29:26.090749 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:26.090755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:26.090767 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:29:26.090798 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:26.090800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:26.090815 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:26.090912 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:26.091562 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:26.092306 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:26.093164 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:26.093614 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:26.093687 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:26.096415 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:26.096426 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:26.096426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:26.096436 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:26.098607 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:26.099042 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:26.099046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:26.099046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:26.099122 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:26.099324 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:26.099325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:26.099325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:26.099471 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:26.099582 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:26.099779 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 16:29:26.099917 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":104,"threads_started":1}
W20260809 16:29:26.100505 31062 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34374
Aug 09 16:29:26 dist-test-slave-j2vl krb5kdc[30826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:29:26 dist-test-slave-j2vl krb5kdc[30826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.129@KRBTEST.COM
I20260809 16:29:26.127807 31666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:26.118909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34388 (local address 127.30.23.129:34203)
0809 16:29:26.119040 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:26.119043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:26.119141 (+ 98us) server_negotiation.cc:407] Connection header received
0809 16:29:26.119257 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:26.119258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:26.119270 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:26.119364 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:26.120032 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:26.120586 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:26.121454 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:26.121759 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:26.121799 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:26.124611 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:26.124613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:26.124613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:26.124624 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:26.126362 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:26.126804 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:26.126806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:26.126806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:26.126871 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:26.127141 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:26.127143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:26.127143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:26.127265 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:26.127376 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:26.127611 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 16:29:26.127704 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 16:29:26.128604 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 30997
I20260809 16:29:26.145991 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 31224
I20260809 16:29:26.160501 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 31446
I20260809 16:29:26.170073 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 30844
2026-08-09T16:29:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2647 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:29:26 dist-test-slave-j2vl krb5kdc[31675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:29:26 dist-test-slave-j2vl krb5kdc[31675](info): set up 2 sockets
Aug 09 16:29:26 dist-test-slave-j2vl krb5kdc[31675](info): commencing operation
krb5kdc: starting...
W20260809 16:29:27.499897 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:29:27 dist-test-slave-j2vl krb5kdc[31675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:29:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:29:27.634424 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:34465
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:45559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:34465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:27.720952 31697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:27.721237 31697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:27.721284 31697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:27.722923 31697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:27.723021 31697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:27.723052 31697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:27.723085 31697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:27.723114 31697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:27.724782 31697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:34465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:34465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:27.725095 31697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:27.725387 31697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:27.730540 31702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:27.730528 31704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:27.730528 31706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:27.730970 31697 server_base.cc:1034] running on GCE node
I20260809 16:29:27.731251 31697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:27.731580 31697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:27.732810 31697 hybrid_clock.cc:648] HybridClock initialized: now 1786292967732779 us; error 53 us; skew 500 ppm
Aug 09 16:29:27 dist-test-slave-j2vl krb5kdc[31675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292967, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:27.739019 31697 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:29:27.739562 31697 webserver.cc:460] Webserver started at http://127.30.23.190:42447/ using document root <none> and password file <none>
I20260809 16:29:27.739801 31697 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:27.739897 31697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:27.740057 31697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:27.741178 31697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "acb472782b654072b8894d42c152cd5f"
format_stamp: "Formatted at 2026-08-09 16:29:27 on dist-test-slave-j2vl"
server_key: "b311b2cd9672d35bf3f82c99ec3a6f0a"
server_key_iv: "68353e55055571f2b156b171830eb1fc"
server_key_version: "encryptionkey@0"
I20260809 16:29:27.741598 31697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "acb472782b654072b8894d42c152cd5f"
format_stamp: "Formatted at 2026-08-09 16:29:27 on dist-test-slave-j2vl"
server_key: "b311b2cd9672d35bf3f82c99ec3a6f0a"
server_key_iv: "68353e55055571f2b156b171830eb1fc"
server_key_version: "encryptionkey@0"
I20260809 16:29:27.743116 31697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:29:27.743979 31717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:27.744205 31697 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:27.744320 31697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "acb472782b654072b8894d42c152cd5f"
format_stamp: "Formatted at 2026-08-09 16:29:27 on dist-test-slave-j2vl"
server_key: "b311b2cd9672d35bf3f82c99ec3a6f0a"
server_key_iv: "68353e55055571f2b156b171830eb1fc"
server_key_version: "encryptionkey@0"
I20260809 16:29:27.744401 31697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:27.783324 31697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:27.785945 31697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:27.786123 31697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:27.790787 31697 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:34465
I20260809 16:29:27.790844 31805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:34465 every 8 connection(s)
I20260809 16:29:27.791235 31697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:27.792449 31806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:27.796579 31806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f: Bootstrap starting.
I20260809 16:29:27.797497 31806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:27.797876 31806 log.cc:824] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:27.798797 31806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f: No bootstrap required, opened a new log
I20260809 16:29:27.800921 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 31697
I20260809 16:29:27.801024 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:27.801182 30814 external_mini_cluster.cc:1428] Setting key 993b98e7bc58f971d9d206b3c6104520
I20260809 16:29:27.801784 31806 raft_consensus.cc:348] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb472782b654072b8894d42c152cd5f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34465 } }
I20260809 16:29:27.801986 31806 raft_consensus.cc:374] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:27.802043 31806 raft_consensus.cc:729] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb472782b654072b8894d42c152cd5f, State: Initialized, Role: FOLLOWER
I20260809 16:29:27.802201 31806 consensus_queue.cc:260] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb472782b654072b8894d42c152cd5f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34465 } }
I20260809 16:29:27.802306 31806 raft_consensus.cc:388] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:27.802348 31806 raft_consensus.cc:482] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:27.802399 31806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:27.803741 31806 raft_consensus.cc:504] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb472782b654072b8894d42c152cd5f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34465 } }
I20260809 16:29:27.803865 31806 leader_election.cc:302] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acb472782b654072b8894d42c152cd5f; no voters:
I20260809 16:29:27.804036 31806 leader_election.cc:288] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:27.804075 31816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:27.804198 31816 raft_consensus.cc:686] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [term 1 LEADER]: Becoming Leader. State: Replica: acb472782b654072b8894d42c152cd5f, State: Running, Role: LEADER
I20260809 16:29:27.804284 31816 consensus_queue.cc:237] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb472782b654072b8894d42c152cd5f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34465 } }
I20260809 16:29:27.804339 31806 sys_catalog.cc:564] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:27.804646 31816 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acb472782b654072b8894d42c152cd5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb472782b654072b8894d42c152cd5f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34465 } } }
I20260809 16:29:27.804723 31816 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:27.804860 31816 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader acb472782b654072b8894d42c152cd5f. Latest consensus state: current_term: 1 leader_uuid: "acb472782b654072b8894d42c152cd5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb472782b654072b8894d42c152cd5f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34465 } } }
I20260809 16:29:27.804930 31816 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f [sys.catalog]: This master's current role is: LEADER
Aug 09 16:29:27 dist-test-slave-j2vl krb5kdc[31675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
W20260809 16:29:27.806290 31836 catalog_manager.cc:1558] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:29:27.814043 31836 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:29:27.814152 31824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:27.814395 31824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:27.816291 31824 catalog_manager.cc:1347] Generated new cluster ID: a8dd13e32ea1456b964c13c7f96ac91b
I20260809 16:29:27.816350 31824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:29:27.818329 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:27.801827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33530 (local address 127.30.23.190:34465)
0809 16:29:27.802129 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:27.802140 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:27.802180 (+ 40us) server_negotiation.cc:407] Connection header received
0809 16:29:27.802362 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:27.802370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:27.802469 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:27.802603 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:27.803510 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:27.805033 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:27.806109 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:27.806699 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:27.806796 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:27.812199 (+ 5403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:27.812203 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:27.812204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:27.812218 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:27.815901 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:27.816493 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:27.816497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:27.816497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:27.816622 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:27.816864 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:27.816865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:27.816865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:27.817192 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:27.817872 (+ 680us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:27.817949 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 16:29:27.818105 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":102,"threads_started":1}
I20260809 16:29:27.853035 31824 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:27.853706 31824 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:27.859417 31824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acb472782b654072b8894d42c152cd5f: Generated new TSK 0
I20260809 16:29:27.859612 31824 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:29:27.897091 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34465
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:45559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:27.985982 31846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:27.986194 31846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:27.986218 31846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:27.987324 31846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:27.987358 31846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:27.987385 31846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:29:27.988822 31846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34465
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:27.989012 31846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:27.989197 31846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:27.994534 31854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:27.994517 31853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:27.994796 31846 server_base.cc:1034] running on GCE node
W20260809 16:29:27.994522 31858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:27.995078 31846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:27.995456 31846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:27.996622 31846 hybrid_clock.cc:648] HybridClock initialized: now 1786292967996604 us; error 51 us; skew 500 ppm
Aug 09 16:29:27 dist-test-slave-j2vl krb5kdc[31675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292967, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:27.999876 31846 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:29:28.000355 31846 webserver.cc:460] Webserver started at http://127.30.23.129:37487/ using document root <none> and password file <none>
I20260809 16:29:28.000614 31846 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:28.000685 31846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:28.000839 31846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:28.001988 31846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "3c6992b95f6b4a25a73d169140d3ca1c"
format_stamp: "Formatted at 2026-08-09 16:29:27 on dist-test-slave-j2vl"
server_key: "bfcf5a7862b9d64125ab03b4eea5bb07"
server_key_iv: "79028ce113451d9a1aa0f664d6955c7b"
server_key_version: "encryptionkey@0"
I20260809 16:29:28.002353 31846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "3c6992b95f6b4a25a73d169140d3ca1c"
format_stamp: "Formatted at 2026-08-09 16:29:27 on dist-test-slave-j2vl"
server_key: "bfcf5a7862b9d64125ab03b4eea5bb07"
server_key_iv: "79028ce113451d9a1aa0f664d6955c7b"
server_key_version: "encryptionkey@0"
I20260809 16:29:28.003809 31846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:29:28.004698 31866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:28.004942 31846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:28.005028 31846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "3c6992b95f6b4a25a73d169140d3ca1c"
format_stamp: "Formatted at 2026-08-09 16:29:27 on dist-test-slave-j2vl"
server_key: "bfcf5a7862b9d64125ab03b4eea5bb07"
server_key_iv: "79028ce113451d9a1aa0f664d6955c7b"
server_key_version: "encryptionkey@0"
I20260809 16:29:28.005118 31846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:28.025589 31846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:28.028087 31846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:28.028242 31846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:28.028528 31846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:28.028868 31846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:28.028924 31846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:28.028975 31846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:28.029003 31846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:28.035003 31846 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:41657
I20260809 16:29:28.035121 32050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:41657 every 8 connection(s)
I20260809 16:29:28.035441 31846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:29:28 dist-test-slave-j2vl krb5kdc[31675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292967, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:28.043944 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 31846
I20260809 16:29:28.044057 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:29:28.044252 30814 external_mini_cluster.cc:1428] Setting key 95e570524893fc6b0f81299ec48f912d
I20260809 16:29:28.046517 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.036270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:53691 (local address 127.30.23.190:34465)
0809 16:29:28.036314 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:28.036316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:28.036531 (+ 215us) server_negotiation.cc:407] Connection header received
0809 16:29:28.037629 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:28.037631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:28.037647 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:28.037704 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:28.038436 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.038992 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.039980 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.040191 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.040257 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.042544 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:28.042545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:28.042546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:28.042553 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:28.044501 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.045137 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.045138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.045139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.045219 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.045645 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.045646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.045646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.045812 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:28.045903 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:28.046241 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 16:29:28.046360 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:29:28.046685 32054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.036223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34465 (local address 127.30.23.129:53691)
0809 16:29:28.036396 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:28.036440 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:28.037528 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:28.037763 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:28.037766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:28.038033 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:28.038260 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.038265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.039082 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.039083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.039880 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.039882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.040274 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.040275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.040377 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.040864 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:28.040872 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:28.042426 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:28.044604 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:28.044614 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:28.044617 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:28.044999 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:28.045426 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:28.045428 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:28.045428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:28.045567 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:28.045936 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:28.045943 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:28.046084 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 16:29:28.046180 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 16:29:28.046405 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260809 16:29:28.047215 32051 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34465
I20260809 16:29:28.047340 32051 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:28.047550 32051 heartbeater.cc:511] Master 127.30.23.190:34465 requested a full tablet report, sending...
I20260809 16:29:28.048143 31747 ts_manager.cc:194] Registered new tserver with Master: 3c6992b95f6b4a25a73d169140d3ca1c (127.30.23.129:41657)
I20260809 16:29:28.048851 31747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:53691
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:29:28.076052 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34465
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:45559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:28.159232 32058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:28.159453 32058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:28.159478 32058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:28.160648 32058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:28.160689 32058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:28.160717 32058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:29:28.162217 32058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34465
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:28.162483 32058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:28.162725 32058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:28.168190 32071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:28.168190 32069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:28.168258 32068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:28.168440 32058 server_base.cc:1034] running on GCE node
I20260809 16:29:28.168699 32058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:28.168956 32058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:28.170135 32058 hybrid_clock.cc:648] HybridClock initialized: now 1786292968170122 us; error 34 us; skew 500 ppm
Aug 09 16:29:28 dist-test-slave-j2vl krb5kdc[31675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292968, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:28.172943 32058 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:29:28.173483 32058 webserver.cc:460] Webserver started at http://127.30.23.130:43073/ using document root <none> and password file <none>
I20260809 16:29:28.173718 32058 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:28.173789 32058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:28.173931 32058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:28.175010 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "0c5fffe26be949eebb8d398280085cf7"
format_stamp: "Formatted at 2026-08-09 16:29:28 on dist-test-slave-j2vl"
server_key: "42842181e23638f60bef838898501d08"
server_key_iv: "426ad9dd8c0d605c82f1b12a8d52f6be"
server_key_version: "encryptionkey@0"
I20260809 16:29:28.175376 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "0c5fffe26be949eebb8d398280085cf7"
format_stamp: "Formatted at 2026-08-09 16:29:28 on dist-test-slave-j2vl"
server_key: "42842181e23638f60bef838898501d08"
server_key_iv: "426ad9dd8c0d605c82f1b12a8d52f6be"
server_key_version: "encryptionkey@0"
I20260809 16:29:28.176791 32058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:29:28.177652 32079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:28.177980 32058 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:28.178092 32058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "0c5fffe26be949eebb8d398280085cf7"
format_stamp: "Formatted at 2026-08-09 16:29:28 on dist-test-slave-j2vl"
server_key: "42842181e23638f60bef838898501d08"
server_key_iv: "426ad9dd8c0d605c82f1b12a8d52f6be"
server_key_version: "encryptionkey@0"
I20260809 16:29:28.178166 32058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:28.199879 32058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:28.202035 32058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:28.202198 32058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:28.202461 32058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:28.202796 32058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:28.202853 32058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:28.202889 32058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:28.202920 32058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:28.209143 32058 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:37093
I20260809 16:29:28.209297 32261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:37093 every 8 connection(s)
I20260809 16:29:28.209619 32058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:29:28.212069 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 32058
I20260809 16:29:28.212168 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:29:28.212335 30814 external_mini_cluster.cc:1428] Setting key 68ae0babc81c12dc21c5a9a2b27a3722
Aug 09 16:29:28 dist-test-slave-j2vl krb5kdc[31675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292968, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:28.221029 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.210452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:49289 (local address 127.30.23.190:34465)
0809 16:29:28.210498 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:28.210500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:28.210746 (+ 246us) server_negotiation.cc:407] Connection header received
0809 16:29:28.211675 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:28.211677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:28.211687 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:28.211752 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:28.212706 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.213295 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.214854 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.215203 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.215285 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.217665 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:28.217667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:28.217668 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:28.217677 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:28.219339 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.219857 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.219857 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.219858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.219922 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.220292 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.220293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.220293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.220432 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:28.220499 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:28.220862 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 16:29:28.220961 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:29:28.221273 32268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.210408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34465 (local address 127.30.23.130:49289)
0809 16:29:28.210601 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:28.210644 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:28.211564 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:28.211805 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:28.211809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:28.212236 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:28.212530 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.212536 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.213414 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.213415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.214737 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.214740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.215322 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.215323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.215445 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.216080 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:28.216089 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:28.217568 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:28.219435 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:28.219443 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:28.219446 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:28.219782 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:28.219992 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:28.219993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:28.219994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:28.220142 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:28.220514 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:28.220521 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:28.220698 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 16:29:28.220828 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 16:29:28.221015 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
I20260809 16:29:28.221880 32263 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34465
I20260809 16:29:28.221993 32263 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:28.222195 32263 heartbeater.cc:511] Master 127.30.23.190:34465 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
I20260809 16:29:28.222605 31747 ts_manager.cc:194] Registered new tserver with Master: 0c5fffe26be949eebb8d398280085cf7 (127.30.23.130:37093)
I20260809 16:29:28.223006 31747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:49289
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:29:28.241068 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34465
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:45559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:28.325014 32271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:28.325225 32271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:28.325250 32271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:28.326496 32271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:28.326545 32271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:28.326571 32271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:29:28.328126 32271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34465
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:28.328334 32271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:28.328622 32271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:28.334152 32280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:28.334139 32285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:28.334139 32279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:28.334473 32271 server_base.cc:1034] running on GCE node
I20260809 16:29:28.334695 32271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:28.334924 32271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:28.336109 32271 hybrid_clock.cc:648] HybridClock initialized: now 1786292968336097 us; error 41 us; skew 500 ppm
Aug 09 16:29:28 dist-test-slave-j2vl krb5kdc[31675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292968, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:28.339913 32271 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:29:28.340416 32271 webserver.cc:460] Webserver started at http://127.30.23.131:39229/ using document root <none> and password file <none>
I20260809 16:29:28.340685 32271 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:28.340724 32271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:28.340910 32271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:28.342109 32271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "dfd84803f04c4960aed22e04857599bb"
format_stamp: "Formatted at 2026-08-09 16:29:28 on dist-test-slave-j2vl"
server_key: "d7d76e02358494f5bbc3d8fd4b70540f"
server_key_iv: "87189e719b059f07f89b31a48a63bf56"
server_key_version: "encryptionkey@0"
I20260809 16:29:28.342473 32271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "dfd84803f04c4960aed22e04857599bb"
format_stamp: "Formatted at 2026-08-09 16:29:28 on dist-test-slave-j2vl"
server_key: "d7d76e02358494f5bbc3d8fd4b70540f"
server_key_iv: "87189e719b059f07f89b31a48a63bf56"
server_key_version: "encryptionkey@0"
I20260809 16:29:28.344118 32271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:29:28.345224 32298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:28.345664 32271 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:28.345902 32271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "dfd84803f04c4960aed22e04857599bb"
format_stamp: "Formatted at 2026-08-09 16:29:28 on dist-test-slave-j2vl"
server_key: "d7d76e02358494f5bbc3d8fd4b70540f"
server_key_iv: "87189e719b059f07f89b31a48a63bf56"
server_key_version: "encryptionkey@0"
I20260809 16:29:28.346005 32271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:28.363348 32271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:28.366925 32271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:28.367141 32271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:28.367550 32271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:28.368109 32271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:28.368177 32271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:28.368227 32271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:28.368258 32271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:28.374842 32271 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:40047
I20260809 16:29:28.374979 32490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:40047 every 8 connection(s)
I20260809 16:29:28.375289 32271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:29:28.377403 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 32271
I20260809 16:29:28.377502 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:29:28.377679 30814 external_mini_cluster.cc:1428] Setting key fdfd44281faebedf91e9f2d7615a7e25
Aug 09 16:29:28 dist-test-slave-j2vl krb5kdc[31675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292968, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:28.386373 32493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.376179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34465 (local address 127.30.23.131:52637)
0809 16:29:28.376385 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:28.376438 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:28.377499 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:28.377760 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:28.377765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:28.378109 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:28.378429 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.378441 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.379204 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.379205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.380046 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.380048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.380433 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.380434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.380700 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.381259 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:28.381268 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:28.382811 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:28.384714 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:28.384722 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:28.384724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:28.384981 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:28.385266 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:28.385267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:28.385268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:28.385468 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:28.385802 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:28.385809 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:28.385948 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 16:29:28.386039 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 16:29:28.386176 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260809 16:29:28.386660 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.376268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:52637 (local address 127.30.23.190:34465)
0809 16:29:28.376400 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:28.376402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:28.376551 (+ 149us) server_negotiation.cc:407] Connection header received
0809 16:29:28.377608 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:28.377609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:28.377624 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:28.377688 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:28.378533 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.379111 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.380143 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.380338 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.380417 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.382911 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:28.382912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:28.382912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:28.382999 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:28.384625 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.385074 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.385075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.385075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.385157 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.385578 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.385579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.385579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.385714 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:28.385787 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:28.386159 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 16:29:28.386551 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260809 16:29:28.387508 32491 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34465
I20260809 16:29:28.387603 32491 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:28.387758 32491 heartbeater.cc:511] Master 127.30.23.190:34465 requested a full tablet report, sending...
I20260809 16:29:28.388238 31747 ts_manager.cc:194] Registered new tserver with Master: dfd84803f04c4960aed22e04857599bb (127.30.23.131:40047)
I20260809 16:29:28.388887 31747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:52637
I20260809 16:29:28.390129 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:29:28.398036 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.391866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33544 (local address 127.30.23.190:34465)
0809 16:29:28.391969 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:28.391971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:28.392001 (+ 30us) server_negotiation.cc:407] Connection header received
0809 16:29:28.392020 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:28.392021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:28.392029 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:28.392070 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:28.392598 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.393110 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.393936 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.394123 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.394180 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.395451 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:28.395456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:28.395456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:28.395463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:28.396751 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.397124 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.397125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.397125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.397194 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.397511 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.397512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.397513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.397683 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:28.397778 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:28.397889 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 16:29:28.397968 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 16:29:28.400621 31747 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:29:28.402580 31747 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:29:28.410192 32510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.405883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:41657 (local address 127.0.0.1:50836)
0809 16:29:28.406302 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:28.406312 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:28.406360 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:28.406642 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:28.406643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:28.406646 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:28.406681 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.406682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.407559 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.407560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.408982 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.408985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.409974 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.409974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.410035 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.410039 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:28.410091 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:29:28.410127 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260809 16:29:28.410264 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.405983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50836 (local address 127.30.23.129:41657)
0809 16:29:28.406184 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:28.406189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:28.406422 (+ 233us) server_negotiation.cc:407] Connection header received
0809 16:29:28.406467 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:28.406469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:28.406486 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:28.406555 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:28.406744 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.407432 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.409082 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.409857 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.409946 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.409997 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:28.410121 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 16:29:28.410175 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260809 16:29:28.410192 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.405620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:40047 (local address 127.0.0.1:38636)
0809 16:29:28.405918 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:28.405965 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:28.406042 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:28.406547 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:28.406548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:28.406551 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:28.406611 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.406613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.407958 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.407960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.409022 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.409025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.409860 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.409861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.409947 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.409963 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 16:29:28.410033 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 16:29:28.410096 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260809 16:29:28.410425 32508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.405721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38636 (local address 127.30.23.131:40047)
0809 16:29:28.406303 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:28.406307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:28.406318 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:29:28.406360 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:28.406362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:28.406378 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:28.406456 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:28.406694 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.407881 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.409126 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.409752 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.409857 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.409923 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:28.410234 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 16:29:28.410282 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":81,"threads_started":1}
I20260809 16:29:28.411230 32374 tablet_service.cc:1467] Processing CreateTablet for tablet f13fe4f7654547a1b3a16d21031351d5 (DEFAULT_TABLE table=test-table [id=9bede03669ea470e9ef69968315edd42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:28.411319 31950 tablet_service.cc:1467] Processing CreateTablet for tablet f13fe4f7654547a1b3a16d21031351d5 (DEFAULT_TABLE table=test-table [id=9bede03669ea470e9ef69968315edd42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:28.411595 32374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f13fe4f7654547a1b3a16d21031351d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:28.411708 31950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f13fe4f7654547a1b3a16d21031351d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:28.414572 32515 tablet_bootstrap.cc:492] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c: Bootstrap starting.
I20260809 16:29:28.414572 32517 tablet_bootstrap.cc:492] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb: Bootstrap starting.
I20260809 16:29:28.414921 32512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.405880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.30.23.130:37093)
0809 16:29:28.406220 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:28.406225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:28.406237 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:29:28.406303 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:28.406306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:28.406328 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:28.406405 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:28.406670 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.407802 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.409159 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.409874 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.409935 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.409991 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:28.410266 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 16:29:28.414823 (+ 4557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":87,"threads_started":1}
I20260809 16:29:28.415098 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.405663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:37093 (local address 127.0.0.1:48272)
0809 16:29:28.406094 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:28.406106 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:28.406166 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:28.406514 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:28.406518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:28.406520 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:28.406599 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.406602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.407904 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.407905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.409094 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.409096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.410028 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.410029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.410126 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.410130 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:28.410181 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:29:28.410223 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":29,"thread_start_us":135,"threads_started":1}
I20260809 16:29:28.415287 32515 tablet_bootstrap.cc:654] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:28.415539 32517 tablet_bootstrap.cc:654] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:28.415913 32515 log.cc:824] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:28.415910 32152 tablet_service.cc:1467] Processing CreateTablet for tablet f13fe4f7654547a1b3a16d21031351d5 (DEFAULT_TABLE table=test-table [id=9bede03669ea470e9ef69968315edd42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:28.416111 32517 log.cc:824] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:28.416273 32152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f13fe4f7654547a1b3a16d21031351d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:28.416800 32515 tablet_bootstrap.cc:492] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c: No bootstrap required, opened a new log
I20260809 16:29:28.416877 32515 ts_tablet_manager.cc:1397] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:29:28.416941 32517 tablet_bootstrap.cc:492] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb: No bootstrap required, opened a new log
I20260809 16:29:28.417016 32517 ts_tablet_manager.cc:1397] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:29:28.418992 32517 raft_consensus.cc:348] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd84803f04c4960aed22e04857599bb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40047 } } peers { permanent_uuid: "0c5fffe26be949eebb8d398280085cf7" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 37093 } } peers { permanent_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41657 } }
I20260809 16:29:28.418992 32515 raft_consensus.cc:348] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd84803f04c4960aed22e04857599bb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40047 } } peers { permanent_uuid: "0c5fffe26be949eebb8d398280085cf7" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 37093 } } peers { permanent_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41657 } }
I20260809 16:29:28.419191 32517 raft_consensus.cc:374] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:28.419191 32515 raft_consensus.cc:374] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:28.419235 32517 raft_consensus.cc:729] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfd84803f04c4960aed22e04857599bb, State: Initialized, Role: FOLLOWER
I20260809 16:29:28.419235 32515 raft_consensus.cc:729] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c6992b95f6b4a25a73d169140d3ca1c, State: Initialized, Role: FOLLOWER
I20260809 16:29:28.419358 32515 consensus_queue.cc:260] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd84803f04c4960aed22e04857599bb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40047 } } peers { permanent_uuid: "0c5fffe26be949eebb8d398280085cf7" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 37093 } } peers { permanent_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41657 } }
I20260809 16:29:28.419371 32517 consensus_queue.cc:260] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd84803f04c4960aed22e04857599bb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40047 } } peers { permanent_uuid: "0c5fffe26be949eebb8d398280085cf7" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 37093 } } peers { permanent_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41657 } }
I20260809 16:29:28.419397 32525 tablet_bootstrap.cc:492] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7: Bootstrap starting.
I20260809 16:29:28.419665 32515 ts_tablet_manager.cc:1428] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:29:28.419665 32517 ts_tablet_manager.cc:1428] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:29:28.419693 32051 heartbeater.cc:503] Master 127.30.23.190:34465 was elected leader, sending a full tablet report...
I20260809 16:29:28.419788 32491 heartbeater.cc:503] Master 127.30.23.190:34465 was elected leader, sending a full tablet report...
I20260809 16:29:28.420395 32525 tablet_bootstrap.cc:654] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:28.421030 32525 log.cc:824] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:28.421916 32525 tablet_bootstrap.cc:492] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7: No bootstrap required, opened a new log
I20260809 16:29:28.421975 32525 ts_tablet_manager.cc:1397] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:29:28.423358 32525 raft_consensus.cc:348] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd84803f04c4960aed22e04857599bb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40047 } } peers { permanent_uuid: "0c5fffe26be949eebb8d398280085cf7" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 37093 } } peers { permanent_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41657 } }
I20260809 16:29:28.423498 32525 raft_consensus.cc:374] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:28.423520 32525 raft_consensus.cc:729] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c5fffe26be949eebb8d398280085cf7, State: Initialized, Role: FOLLOWER
I20260809 16:29:28.423643 32525 consensus_queue.cc:260] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd84803f04c4960aed22e04857599bb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40047 } } peers { permanent_uuid: "0c5fffe26be949eebb8d398280085cf7" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 37093 } } peers { permanent_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41657 } }
I20260809 16:29:28.423936 32525 ts_tablet_manager.cc:1428] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:29:28.424006 32263 heartbeater.cc:503] Master 127.30.23.190:34465 was elected leader, sending a full tablet report...
W20260809 16:29:28.460238 32264 tablet.cc:2369] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:29:28.534508 32528 raft_consensus.cc:482] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:29:28.534654 32528 raft_consensus.cc:504] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd84803f04c4960aed22e04857599bb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40047 } } peers { permanent_uuid: "0c5fffe26be949eebb8d398280085cf7" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 37093 } } peers { permanent_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41657 } }
I20260809 16:29:28.535190 32528 leader_election.cc:288] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfd84803f04c4960aed22e04857599bb (127.30.23.131:40047), 0c5fffe26be949eebb8d398280085cf7 (127.30.23.130:37093)
W20260809 16:29:28.536161 32053 tablet.cc:2369] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:29:28.538288 32054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.535448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:37093 (local address 127.30.23.129:44431)
0809 16:29:28.535509 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:28.535520 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:28.535571 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:28.535792 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:28.535793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:28.535795 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:28.535869 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.535871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.536597 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.536599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.537428 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.537431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.538011 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.538013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.538095 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.538099 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:28.538165 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:29:28.538210 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":36}
I20260809 16:29:28.538288 32512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.535519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:44431 (local address 127.30.23.130:37093)
0809 16:29:28.535564 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:28.535566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:28.535577 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:29:28.535668 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:28.535669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:28.535678 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:28.535759 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:28.535947 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.536507 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.537537 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.537926 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.538014 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.538024 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:28.538183 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 16:29:28.538230 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:29:28.538453 32508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.535525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:35513 (local address 127.30.23.131:40047)
0809 16:29:28.535596 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:28.535598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:28.535728 (+ 130us) server_negotiation.cc:407] Connection header received
0809 16:29:28.535783 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:28.535784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:28.535796 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:28.535871 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:28.536036 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.536618 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.537722 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.538102 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.538190 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.538199 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:28.538349 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 16:29:28.538389 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:29:28.538467 32532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.535448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:40047 (local address 127.30.23.129:35513)
0809 16:29:28.535636 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:28.535645 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:28.535731 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:28.535876 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:28.535878 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:28.535879 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:28.535958 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.535960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.536709 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.536711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.537632 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:28.537635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.538191 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.538192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:28.538267 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.538271 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:28.538341 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 16:29:28.538381 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":16,"thread_start_us":93,"threads_started":1}
I20260809 16:29:28.538865 32186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f13fe4f7654547a1b3a16d21031351d5" candidate_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c5fffe26be949eebb8d398280085cf7" is_pre_election: true
I20260809 16:29:28.539003 32186 raft_consensus.cc:2454] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c6992b95f6b4a25a73d169140d3ca1c in term 0.
I20260809 16:29:28.538939 32410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f13fe4f7654547a1b3a16d21031351d5" candidate_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfd84803f04c4960aed22e04857599bb" is_pre_election: true
I20260809 16:29:28.539149 32410 raft_consensus.cc:2454] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c6992b95f6b4a25a73d169140d3ca1c in term 0.
I20260809 16:29:28.539281 31871 leader_election.cc:302] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c5fffe26be949eebb8d398280085cf7, 3c6992b95f6b4a25a73d169140d3ca1c; no voters:
I20260809 16:29:28.539464 32528 raft_consensus.cc:2781] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:29:28.539633 32528 raft_consensus.cc:482] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:29:28.539691 32528 raft_consensus.cc:3037] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:28.540505 32528 raft_consensus.cc:504] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd84803f04c4960aed22e04857599bb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40047 } } peers { permanent_uuid: "0c5fffe26be949eebb8d398280085cf7" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 37093 } } peers { permanent_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41657 } }
I20260809 16:29:28.540727 32528 leader_election.cc:288] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [CANDIDATE]: Term 1 election: Requested vote from peers dfd84803f04c4960aed22e04857599bb (127.30.23.131:40047), 0c5fffe26be949eebb8d398280085cf7 (127.30.23.130:37093)
I20260809 16:29:28.540868 32410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f13fe4f7654547a1b3a16d21031351d5" candidate_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfd84803f04c4960aed22e04857599bb"
I20260809 16:29:28.540897 32186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f13fe4f7654547a1b3a16d21031351d5" candidate_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c5fffe26be949eebb8d398280085cf7"
I20260809 16:29:28.540980 32410 raft_consensus.cc:3037] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:28.541018 32186 raft_consensus.cc:3037] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:28.541859 32410 raft_consensus.cc:2454] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c6992b95f6b4a25a73d169140d3ca1c in term 1.
I20260809 16:29:28.541875 32186 raft_consensus.cc:2454] T f13fe4f7654547a1b3a16d21031351d5 P 0c5fffe26be949eebb8d398280085cf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c6992b95f6b4a25a73d169140d3ca1c in term 1.
I20260809 16:29:28.542124 31871 leader_election.cc:302] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c5fffe26be949eebb8d398280085cf7, 3c6992b95f6b4a25a73d169140d3ca1c; no voters:
I20260809 16:29:28.542256 32528 raft_consensus.cc:2781] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:28.542464 32528 raft_consensus.cc:686] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [term 1 LEADER]: Becoming Leader. State: Replica: 3c6992b95f6b4a25a73d169140d3ca1c, State: Running, Role: LEADER
I20260809 16:29:28.542577 32528 consensus_queue.cc:237] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd84803f04c4960aed22e04857599bb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40047 } } peers { permanent_uuid: "0c5fffe26be949eebb8d398280085cf7" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 37093 } } peers { permanent_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41657 } }
I20260809 16:29:28.543566 31747 catalog_manager.cc:5495] T f13fe4f7654547a1b3a16d21031351d5 P 3c6992b95f6b4a25a73d169140d3ca1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c6992b95f6b4a25a73d169140d3ca1c (127.30.23.129). New cstate: current_term: 1 leader_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd84803f04c4960aed22e04857599bb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c5fffe26be949eebb8d398280085cf7" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 37093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c6992b95f6b4a25a73d169140d3ca1c" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41657 } health_report { overall_health: HEALTHY } } }
Aug 09 16:29:28 dist-test-slave-j2vl krb5kdc[31675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.129@KRBTEST.COM
I20260809 16:29:28.561815 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.553062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50844 (local address 127.30.23.129:41657)
0809 16:29:28.553146 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:28.553149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:28.553195 (+ 46us) server_negotiation.cc:407] Connection header received
0809 16:29:28.553244 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:28.553245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:28.553257 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:28.553322 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:28.554054 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.554536 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.555352 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.555560 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.555628 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.558230 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:28.558241 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:28.558241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:28.558258 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:28.560207 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.560710 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.560713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.560714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.560786 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.561098 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.561099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.561099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.561267 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:28.561487 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:28.561616 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 16:29:28.561689 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 16:29:28 dist-test-slave-j2vl krb5kdc[31675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292968, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:29:28 dist-test-slave-j2vl krb5kdc[31675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292968, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.23.129@KRBTEST.COM
I20260809 16:29:28.588327 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.580390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50860 (local address 127.30.23.129:41657)
0809 16:29:28.580465 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:28.580467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:28.580723 (+ 256us) server_negotiation.cc:407] Connection header received
0809 16:29:28.580770 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:28.580771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:28.580782 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:28.580826 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:28.581510 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.582027 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.582792 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.583017 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.583083 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.585418 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:28.585420 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:28.585420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:28.585435 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:28.586897 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.587254 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.587255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.587256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.587317 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.587557 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.587558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.587558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.587670 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:28.587781 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:28.588069 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 16:29:28.588202 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 16:29:28 dist-test-slave-j2vl krb5kdc[31675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:29:28 dist-test-slave-j2vl krb5kdc[31675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.129@KRBTEST.COM
I20260809 16:29:28.614223 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:28.606567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50864 (local address 127.30.23.129:41657)
0809 16:29:28.606670 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:28.606672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:28.606799 (+ 127us) server_negotiation.cc:407] Connection header received
0809 16:29:28.606900 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:28.606901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:28.606935 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:28.606999 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:28.607632 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.608140 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.608877 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:28.609080 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:28.609170 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:28.611732 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:28.611734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:28.611735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:28.611744 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:28.613028 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.613411 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.613412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.613412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.613478 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:28.613693 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:28.613694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:28.613695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:28.613803 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:28.613925 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:28.614026 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 16:29:28.614112 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 16:29:28.614753 31907 tablet_service.cc:2908] Rejecting scan request for tablet f13fe4f7654547a1b3a16d21031351d5: Uninitialized: clean time has not yet been initialized
I20260809 16:29:28.615778 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 31846
W20260809 16:29:28.625982 32492 tablet.cc:2369] T f13fe4f7654547a1b3a16d21031351d5 P dfd84803f04c4960aed22e04857599bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:29:28.628229 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 32058
I20260809 16:29:28.640800 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 32271
I20260809 16:29:28.652603 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 31697
2026-08-09T16:29:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2480 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:29:28 dist-test-slave-j2vl krb5kdc[32555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:29:28 dist-test-slave-j2vl krb5kdc[32555](info): set up 2 sockets
Aug 09 16:29:28 dist-test-slave-j2vl krb5kdc[32555](info): commencing operation
krb5kdc: starting...
W20260809 16:29:29.974802 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:29:30 dist-test-slave-j2vl krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:29:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:29:30.109366 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:38081
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:38081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:30.197216 32574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:30.197531 32574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:30.197577 32574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:30.198858 32574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:30.198939 32574 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 16:29:30.198962 32574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:30.198987 32574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:30.199014 32574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:30.199038 32574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:30.200980 32574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:38081
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:38081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:30.201324 32574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:30.201622 32574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:30.207356 32586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:30.207342 32589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:30.207398 32585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:30.207906 32574 server_base.cc:1034] running on GCE node
I20260809 16:29:30.208258 32574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:30.208595 32574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:30.209792 32574 hybrid_clock.cc:648] HybridClock initialized: now 1786292970209770 us; error 40 us; skew 500 ppm
Aug 09 16:29:30 dist-test-slave-j2vl krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292970, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:30.212975 32574 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:29:30.213599 32574 webserver.cc:460] Webserver started at http://127.30.23.190:37253/ using document root <none> and password file <none>
I20260809 16:29:30.213977 32574 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:30.214201 32574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:30.214412 32574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:30.215691 32574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "0b59439fa4ab4e12bd2f4c6d1f4f8726"
format_stamp: "Formatted at 2026-08-09 16:29:30 on dist-test-slave-j2vl"
server_key: "8689b2140b0551116cd7acff68bce240"
server_key_iv: "6562889c6ce9afc9c63771f276ee4a94"
server_key_version: "encryptionkey@0"
I20260809 16:29:30.216118 32574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "0b59439fa4ab4e12bd2f4c6d1f4f8726"
format_stamp: "Formatted at 2026-08-09 16:29:30 on dist-test-slave-j2vl"
server_key: "8689b2140b0551116cd7acff68bce240"
server_key_iv: "6562889c6ce9afc9c63771f276ee4a94"
server_key_version: "encryptionkey@0"
I20260809 16:29:30.217921 32574 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260809 16:29:30.218918 32596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:30.219216 32574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:30.219564 32574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "0b59439fa4ab4e12bd2f4c6d1f4f8726"
format_stamp: "Formatted at 2026-08-09 16:29:30 on dist-test-slave-j2vl"
server_key: "8689b2140b0551116cd7acff68bce240"
server_key_iv: "6562889c6ce9afc9c63771f276ee4a94"
server_key_version: "encryptionkey@0"
I20260809 16:29:30.219673 32574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:30.252123 32574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:30.254887 32574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:30.255082 32574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:30.260716 32574 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:38081
I20260809 16:29:30.260926 32695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:38081 every 8 connection(s)
I20260809 16:29:30.261230 32574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:30.261996 32697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:30.264369 32697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726: Bootstrap starting.
I20260809 16:29:30.264947 32697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:30.265192 32697 log.cc:824] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:30.266023 32697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726: No bootstrap required, opened a new log
I20260809 16:29:30.267405 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 32574
I20260809 16:29:30.267515 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:30.267675 30814 external_mini_cluster.cc:1428] Setting key aca3983e212f7b3b46fd86d54296c86a
I20260809 16:29:30.268852 32697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b59439fa4ab4e12bd2f4c6d1f4f8726" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38081 } }
I20260809 16:29:30.269033 32697 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:30.269080 32697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b59439fa4ab4e12bd2f4c6d1f4f8726, State: Initialized, Role: FOLLOWER
I20260809 16:29:30.269219 32697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b59439fa4ab4e12bd2f4c6d1f4f8726" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38081 } }
I20260809 16:29:30.269325 32697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:30.269402 32697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:30.269466 32697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:30.270098 32697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b59439fa4ab4e12bd2f4c6d1f4f8726" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38081 } }
I20260809 16:29:30.270228 32697 leader_election.cc:302] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b59439fa4ab4e12bd2f4c6d1f4f8726; no voters:
I20260809 16:29:30.270483 32703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:30.270506 32697 leader_election.cc:288] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:30.270655 32703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [term 1 LEADER]: Becoming Leader. State: Replica: 0b59439fa4ab4e12bd2f4c6d1f4f8726, State: Running, Role: LEADER
I20260809 16:29:30.270747 32703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b59439fa4ab4e12bd2f4c6d1f4f8726" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38081 } }
I20260809 16:29:30.270926 32697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:30.271067 32703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b59439fa4ab4e12bd2f4c6d1f4f8726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b59439fa4ab4e12bd2f4c6d1f4f8726" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38081 } } }
I20260809 16:29:30.271145 32703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:30.271078 32708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b59439fa4ab4e12bd2f4c6d1f4f8726. Latest consensus state: current_term: 1 leader_uuid: "0b59439fa4ab4e12bd2f4c6d1f4f8726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b59439fa4ab4e12bd2f4c6d1f4f8726" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38081 } } }
I20260809 16:29:30.271325 32708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:30.272060 32710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:30.272600 32710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:30.274236 32710 catalog_manager.cc:1347] Generated new cluster ID: 07e9f431fb314d4ea568c98085ba0fb2
I20260809 16:29:30.274286 32710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:29:30 dist-test-slave-j2vl krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:30.282447 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.268287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33034 (local address 127.30.23.190:38081)
0809 16:29:30.268571 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.268579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.268609 (+ 30us) server_negotiation.cc:407] Connection header received
0809 16:29:30.268828 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.268834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.268885 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.269038 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:30.269878 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.271179 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.273233 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.273874 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.273995 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.277207 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:30.277215 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:30.277215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:30.277231 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:30.279470 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.280007 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.280011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.280011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.280190 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.280471 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.280472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.280473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.280840 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:30.281165 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.281874 (+ 709us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.282173 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
I20260809 16:29:30.289178 32710 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:30.290058 32710 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:30.293256 32710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726: Generated new TSK 0
I20260809 16:29:30.293510 32710 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:29:30.330883 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:38081
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:30.418491 32740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:30.418753 32740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:30.418798 32740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:30.420102 32740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:30.420171 32740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:30.420207 32740 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:29:30.420238 32740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:29:30.421819 32740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.23.190:38081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:30.422111 32740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:30.422361 32740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:29:30.427758 32740 server_base.cc:1034] running on GCE node
W20260809 16:29:30.427743 32754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:30.427856 32749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:30.427759 32750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:30.428165 32740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:30.428373 32740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:30.429508 32740 hybrid_clock.cc:648] HybridClock initialized: now 1786292970429477 us; error 43 us; skew 500 ppm
Aug 09 16:29:30 dist-test-slave-j2vl krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292970, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:30.432474 32740 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:29:30.433171 32740 webserver.cc:460] Webserver started at http://127.30.23.129:39701/ using document root <none> and password file <none>
I20260809 16:29:30.433501 32740 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:30.433571 32740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:30.433714 32740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:30.434835 32740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "ad7385e45cd54214b9672c92562fb3e0"
format_stamp: "Formatted at 2026-08-09 16:29:30 on dist-test-slave-j2vl"
server_key: "931f224b3ab30973429b7138bc356487"
server_key_iv: "fe6a8462100480552193f054178ee05f"
server_key_version: "encryptionkey@0"
I20260809 16:29:30.435226 32740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "ad7385e45cd54214b9672c92562fb3e0"
format_stamp: "Formatted at 2026-08-09 16:29:30 on dist-test-slave-j2vl"
server_key: "931f224b3ab30973429b7138bc356487"
server_key_iv: "fe6a8462100480552193f054178ee05f"
server_key_version: "encryptionkey@0"
I20260809 16:29:30.436666 32740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:29:30.437547 32763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:30.437757 32740 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:30.437885 32740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "ad7385e45cd54214b9672c92562fb3e0"
format_stamp: "Formatted at 2026-08-09 16:29:30 on dist-test-slave-j2vl"
server_key: "931f224b3ab30973429b7138bc356487"
server_key_iv: "fe6a8462100480552193f054178ee05f"
server_key_version: "encryptionkey@0"
I20260809 16:29:30.437968 32740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:30.452401 32740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:30.454597 32740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:30.454780 32740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:30.455302 32740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:30.455360 32740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:30.455397 32740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:30.455427 32740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:30.462905 32740 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:46083
I20260809 16:29:30.463066 475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:46083 every 8 connection(s)
Aug 09 16:29:30 dist-test-slave-j2vl krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292970, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:30.463349 32740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:29:30.467552 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 32740
I20260809 16:29:30.467653 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:29:30.467840 311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.456060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38081 (local address 127.30.23.129:42157)
0809 16:29:30.456243 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:30.456278 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:30.456889 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:30.457223 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:30.457228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:30.457663 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:30.457960 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.457968 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.458683 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.458684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.460133 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.460137 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.460596 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.460597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.460761 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.461273 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:30.461285 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:30.463859 (+ 2574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:30.466135 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:30.466159 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:30.466162 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:30.466444 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:30.466671 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:30.466672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:30.466672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:30.466788 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:30.467084 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:30.467093 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:30.467271 (+ 178us) client_negotiation.cc:769] Sending connection context
0809 16:29:30.467390 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 16:29:30.467584 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
I20260809 16:29:30.467847 30814 external_mini_cluster.cc:1428] Setting key b93508611099235968b15b12961f4ead
I20260809 16:29:30.468681 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.456076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:42157 (local address 127.30.23.190:38081)
0809 16:29:30.456126 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.456133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.456384 (+ 251us) server_negotiation.cc:407] Connection header received
0809 16:29:30.457016 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.457019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.457033 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.457110 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:30.458088 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.458608 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.460248 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.460502 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.460574 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.464095 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:30.464097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:30.464098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:30.464110 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:30.466044 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.466528 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.466529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.466530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.466592 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.466872 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.466873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.466873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.467020 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:30.467087 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.468459 (+ 1372us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.468572 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:29:30.469053 476 heartbeater.cc:347] Connected to a master server at 127.30.23.190:38081
I20260809 16:29:30.469153 476 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:30.469378 476 heartbeater.cc:511] Master 127.30.23.190:38081 requested a full tablet report, sending...
I20260809 16:29:30.469959 32625 ts_manager.cc:194] Registered new tserver with Master: ad7385e45cd54214b9672c92562fb3e0 (127.30.23.129:46083)
I20260809 16:29:30.470822 32625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:42157
I20260809 16:29:30.476791 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.470309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:40849 (local address 127.30.23.190:38081)
0809 16:29:30.470352 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.470355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.470712 (+ 357us) server_negotiation.cc:407] Connection header received
0809 16:29:30.470820 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.470822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.470833 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.470877 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:30.471308 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.472033 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.472781 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.473001 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.473068 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.474185 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:30.474186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:30.474186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:30.474193 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:30.475586 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.476024 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.476025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.476026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.476095 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.476315 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.476316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.476316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.476428 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:30.476483 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.476657 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.476727 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:29:30.476866 486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.470388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38081 (local address 127.30.23.129:40849)
0809 16:29:30.470639 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:30.470648 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:30.470725 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:30.470891 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:30.470893 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:30.471136 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:30.471209 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.471212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.472161 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.472162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.472711 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.472712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.473090 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.473091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.473173 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.473652 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:30.473657 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:30.474094 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:30.475674 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:30.475675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:30.475675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:30.475931 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:30.476173 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:30.476173 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:30.476174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:30.476247 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:30.476499 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:30.476500 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:30.476555 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 16:29:30.476643 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:29:30.476756 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:29:30.497001 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:38081
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:30.581801 491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:30.582065 491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:30.582110 491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:30.583242 491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:30.583277 491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:30.583297 491 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:29:30.583315 491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:29:30.584846 491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.23.190:38081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:30.585107 491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:30.585399 491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:30.590914 501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:30.590891 506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:30.590891 500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:30.590924 491 server_base.cc:1034] running on GCE node
I20260809 16:29:30.591419 491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:30.591676 491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:30.592895 491 hybrid_clock.cc:648] HybridClock initialized: now 1786292970592857 us; error 67 us; skew 500 ppm
Aug 09 16:29:30 dist-test-slave-j2vl krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292970, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:30.595813 491 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:29:30.596370 491 webserver.cc:460] Webserver started at http://127.30.23.130:45015/ using document root <none> and password file <none>
I20260809 16:29:30.596632 491 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:30.596702 491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:30.596853 491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:30.598028 491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "358bfead002847ddbddf9fd55abf839b"
format_stamp: "Formatted at 2026-08-09 16:29:30 on dist-test-slave-j2vl"
server_key: "482f97ae5b860c0e70c6afdf08faf7fb"
server_key_iv: "75a177d7e703f7a756da86a22fdb3b58"
server_key_version: "encryptionkey@0"
I20260809 16:29:30.598408 491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "358bfead002847ddbddf9fd55abf839b"
format_stamp: "Formatted at 2026-08-09 16:29:30 on dist-test-slave-j2vl"
server_key: "482f97ae5b860c0e70c6afdf08faf7fb"
server_key_iv: "75a177d7e703f7a756da86a22fdb3b58"
server_key_version: "encryptionkey@0"
I20260809 16:29:30.599843 491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:29:30.600695 516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:30.601025 491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:30.601243 491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "358bfead002847ddbddf9fd55abf839b"
format_stamp: "Formatted at 2026-08-09 16:29:30 on dist-test-slave-j2vl"
server_key: "482f97ae5b860c0e70c6afdf08faf7fb"
server_key_iv: "75a177d7e703f7a756da86a22fdb3b58"
server_key_version: "encryptionkey@0"
I20260809 16:29:30.601326 491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:30.640022 491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:30.642455 491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:30.642617 491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:30.643281 491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:30.643335 491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:30.643378 491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:30.643391 491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:29:30 dist-test-slave-j2vl krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292970, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:30.650923 491 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:42477
I20260809 16:29:30.651327 491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:29:30.651291 716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:42477 every 8 connection(s)
I20260809 16:29:30.653244 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 491
I20260809 16:29:30.653323 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:29:30.653561 30814 external_mini_cluster.cc:1428] Setting key 6205bd8471ac26245aec85f522d0ddd1
I20260809 16:29:30.654513 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.643724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:52731 (local address 127.30.23.190:38081)
0809 16:29:30.643801 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.643804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.644059 (+ 255us) server_negotiation.cc:407] Connection header received
0809 16:29:30.644553 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.644554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.644569 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.644655 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:30.645560 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.646407 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.647490 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.647696 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.647788 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.650593 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:30.650596 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:30.650596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:30.650606 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:30.652634 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.653192 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.653194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.653194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.653262 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.653682 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.653684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.653685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.653846 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:30.653912 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.654339 (+ 427us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.654447 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 16:29:30.654691 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.643702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38081 (local address 127.30.23.130:52731)
0809 16:29:30.643921 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:30.643963 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:30.644455 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:30.644740 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:30.644744 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:30.645169 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:30.645476 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.645481 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.646542 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.646544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.647381 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.647384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.647830 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.647831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.647960 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.648617 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:30.648626 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:30.650477 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:30.652783 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:30.652791 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:30.652794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:30.653069 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:30.653417 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:30.653419 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:30.653419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:30.653571 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:30.653910 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:30.653917 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:30.654110 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 16:29:30.654229 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 16:29:30.654447 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":136,"threads_started":1}
I20260809 16:29:30.656817 717 heartbeater.cc:347] Connected to a master server at 127.30.23.190:38081
I20260809 16:29:30.656944 717 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:30.657145 717 heartbeater.cc:511] Master 127.30.23.190:38081 requested a full tablet report, sending...
I20260809 16:29:30.657594 32625 ts_manager.cc:194] Registered new tserver with Master: 358bfead002847ddbddf9fd55abf839b (127.30.23.130:42477)
I20260809 16:29:30.658039 32625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:52731
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
I20260809 16:29:30.665225 729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.658222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38081 (local address 127.30.23.130:35677)
0809 16:29:30.658397 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:30.658410 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:30.658483 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:30.658618 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:30.658619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:30.658871 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:30.658946 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.658949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.659802 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.659803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.660810 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.660814 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.661200 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.661202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.661295 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.661842 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:30.661850 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:30.662305 (+ 455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:30.664058 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:30.664059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:30.664059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:30.664317 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:30.664536 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:30.664537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:30.664537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:30.664615 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:30.664873 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:30.664873 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:30.664920 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 16:29:30.664987 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:29:30.665082 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260809 16:29:30.665198 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.658218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:35677 (local address 127.30.23.190:38081)
0809 16:29:30.658268 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.658271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.658487 (+ 216us) server_negotiation.cc:407] Connection header received
0809 16:29:30.658540 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.658541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.658552 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.658614 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:30.659089 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.659675 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.660923 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.661111 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.661176 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.662409 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:30.662410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:30.662410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:30.662417 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:30.663965 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.664390 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.664392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.664392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.664454 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.664667 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.664668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.664668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.664813 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:30.664888 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.665052 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.665131 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:29:30.689061 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:38081
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:30.774083 731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:30.774338 731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:30.774385 731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:30.775677 731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:30.775763 731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:30.775804 731 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:29:30.775863 731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:29:30.777446 731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.23.190:38081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:30.777751 731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:30.778007 731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:30.783573 751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:30.783576 742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:30.783548 743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:30.783854 731 server_base.cc:1034] running on GCE node
I20260809 16:29:30.784273 731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:30.784566 731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:30.785717 731 hybrid_clock.cc:648] HybridClock initialized: now 1786292970785696 us; error 37 us; skew 500 ppm
Aug 09 16:29:30 dist-test-slave-j2vl krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292970, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:30.788530 731 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:29:30.789006 731 webserver.cc:460] Webserver started at http://127.30.23.131:36297/ using document root <none> and password file <none>
I20260809 16:29:30.789255 731 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:30.789304 731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:30.789507 731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:30.790625 731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "e69098f09c4e4ba391b4eea0bcb010d8"
format_stamp: "Formatted at 2026-08-09 16:29:30 on dist-test-slave-j2vl"
server_key: "09fec967b53b3cd9101be4991d5707ce"
server_key_iv: "5be6447f0ea2537eb7a62ed8e81a1df6"
server_key_version: "encryptionkey@0"
I20260809 16:29:30.790992 731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "e69098f09c4e4ba391b4eea0bcb010d8"
format_stamp: "Formatted at 2026-08-09 16:29:30 on dist-test-slave-j2vl"
server_key: "09fec967b53b3cd9101be4991d5707ce"
server_key_iv: "5be6447f0ea2537eb7a62ed8e81a1df6"
server_key_version: "encryptionkey@0"
I20260809 16:29:30.792454 731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:29:30.793367 763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:30.793591 731 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:30.793684 731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "e69098f09c4e4ba391b4eea0bcb010d8"
format_stamp: "Formatted at 2026-08-09 16:29:30 on dist-test-slave-j2vl"
server_key: "09fec967b53b3cd9101be4991d5707ce"
server_key_iv: "5be6447f0ea2537eb7a62ed8e81a1df6"
server_key_version: "encryptionkey@0"
I20260809 16:29:30.793756 731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:30.812309 731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:30.814872 731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:30.815068 731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:30.815688 731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:30.815730 731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:30.815759 731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:30.815774 731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:30.823313 731 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:41253
I20260809 16:29:30.823385 958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:41253 every 8 connection(s)
Aug 09 16:29:30 dist-test-slave-j2vl krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292970, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:30.823752 731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:29:30.825165 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 731
I20260809 16:29:30.825268 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:29:30.825492 30814 external_mini_cluster.cc:1428] Setting key 23d4e34d9f1116f33a31ceb3377d2de4
I20260809 16:29:30.827625 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.816383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:41573 (local address 127.30.23.190:38081)
0809 16:29:30.816460 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.816462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.816716 (+ 254us) server_negotiation.cc:407] Connection header received
0809 16:29:30.817277 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.817278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.817300 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.817404 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:30.818378 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.818900 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.820557 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.820824 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.820906 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.824315 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:30.824317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:30.824317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:30.824342 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:30.826086 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.826527 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.826529 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.826529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.826598 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.826866 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.826867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.826867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.827001 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:30.827078 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.827366 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.827529 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 16:29:30.827862 775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.816352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38081 (local address 127.30.23.131:41573)
0809 16:29:30.816580 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:30.816625 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:30.817162 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:30.817506 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:30.817512 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:30.817931 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:30.818254 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.818261 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.819003 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.819004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.820429 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.820433 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.820941 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.820942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.821068 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.822289 (+ 1221us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:30.822300 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:30.824219 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:30.826187 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:30.826194 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:30.826196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:30.826434 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:30.826671 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:30.826671 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:30.826672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:30.826788 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:30.827069 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:30.827076 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:30.827220 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 16:29:30.827330 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 16:29:30.827590 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":133,"threads_started":1}
I20260809 16:29:30.828369 959 heartbeater.cc:347] Connected to a master server at 127.30.23.190:38081
I20260809 16:29:30.828462 959 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:30.828649 959 heartbeater.cc:511] Master 127.30.23.190:38081 requested a full tablet report, sending...
I20260809 16:29:30.829012 32625 ts_manager.cc:194] Registered new tserver with Master: e69098f09c4e4ba391b4eea0bcb010d8 (127.30.23.131:41253)
I20260809 16:29:30.829521 32625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:41573
I20260809 16:29:30.834975 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.829490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:50659 (local address 127.30.23.190:38081)
0809 16:29:30.829536 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.829538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.829636 (+ 98us) server_negotiation.cc:407] Connection header received
0809 16:29:30.829700 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.829701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.829720 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.829779 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:30.830152 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.830678 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.831426 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.831645 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.831719 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.832579 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:30.832580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:30.832581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:30.832587 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:30.834042 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.834389 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.834390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.834390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.834441 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.834603 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.834604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.834604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.834691 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:30.834753 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.834857 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.834913 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:29:30.835067 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.829418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38081 (local address 127.30.23.131:50659)
0809 16:29:30.829543 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:30.829555 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:30.829635 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:30.829801 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:30.829802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:30.830017 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:30.830071 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.830072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.830793 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.830794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.831306 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.831308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.831726 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.831726 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.831785 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.832184 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:30.832188 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:30.832478 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:30.834146 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:30.834147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:30.834147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:30.834319 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:30.834501 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:30.834502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:30.834502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:30.834556 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:30.834755 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:30.834755 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:30.834793 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 16:29:30.834843 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:29:30.834954 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260809 16:29:30.837623 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:29:30 dist-test-slave-j2vl krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292970, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:29:30 dist-test-slave-j2vl krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292970, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:30.865401 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.857019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33046 (local address 127.30.23.190:38081)
0809 16:29:30.857200 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.857202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.857221 (+ 19us) server_negotiation.cc:407] Connection header received
0809 16:29:30.857249 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.857250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.857260 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.857325 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:30.858187 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.858719 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.859630 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.859823 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.859888 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.862414 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:30.862415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:30.862415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:30.862423 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:30.863820 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.864239 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.864241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.864241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.864324 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:30.864679 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:30.864680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:30.864680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:30.864813 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:30.864905 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.865080 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.865158 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260809 16:29:30.867339 32625 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:29:30.869097 32625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:29:30.877286 996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.872646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48808 (local address 127.30.23.131:41253)
0809 16:29:30.872854 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.872859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.872869 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:29:30.873125 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.873129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.873146 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.873226 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:30.873468 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.874773 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.876137 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.876841 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.876916 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.876966 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.877137 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.877178 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
I20260809 16:29:30.877305 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.872196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:41253 (local address 127.0.0.1:48808)
0809 16:29:30.872395 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:30.872437 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:30.872508 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:30.873257 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:30.873258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:30.873261 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:30.873323 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.873325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.874857 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.874858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.876048 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.876050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.876940 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.876940 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.877019 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.877022 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:29:30.877115 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 16:29:30.877187 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":31,"thread_start_us":86,"threads_started":1}
I20260809 16:29:30.877305 990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.872468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:42477 (local address 127.0.0.1:52386)
0809 16:29:30.873012 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:30.873023 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:30.873068 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:30.873552 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:30.873553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:30.873555 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:30.873590 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.873591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.875002 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.875003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.875866 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.875868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.876859 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.876860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.876998 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.877018 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 16:29:30.877115 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 16:29:30.877187 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260809 16:29:30.877638 995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.872577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52386 (local address 127.30.23.130:42477)
0809 16:29:30.873397 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.873402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.873412 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:29:30.873454 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.873456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.873471 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.873557 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:30.873655 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.874922 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.875972 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.876742 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.876830 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.876916 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.877463 (+ 547us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.877521 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":102,"threads_started":1}
I20260809 16:29:30.878297 849 tablet_service.cc:1467] Processing CreateTablet for tablet 15da0e418e4347d680fa999b7c0a4d8e (DEFAULT_TABLE table=test-table [id=ed2945e8fa7c4a97ba61d506e3ef8411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:30.878424 600 tablet_service.cc:1467] Processing CreateTablet for tablet 15da0e418e4347d680fa999b7c0a4d8e (DEFAULT_TABLE table=test-table [id=ed2945e8fa7c4a97ba61d506e3ef8411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:30.878726 849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15da0e418e4347d680fa999b7c0a4d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:30.878845 600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15da0e418e4347d680fa999b7c0a4d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:30.878964 994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.872224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:46083 (local address 127.0.0.1:39556)
0809 16:29:30.872612 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:30.872625 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:30.872682 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:30.873098 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:30.873101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:30.873104 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:30.873196 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.873199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.874245 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.874246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.876047 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.876050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.878665 (+ 2615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.878666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.878752 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.878758 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:29:30.878829 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 16:29:30.878882 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":223,"thread_start_us":108,"threads_started":1}
I20260809 16:29:30.879012 992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.872332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39556 (local address 127.30.23.129:46083)
0809 16:29:30.872656 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.872663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.872676 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:29:30.872904 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.872907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.872926 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.872998 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:30.873275 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.874158 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.877009 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.878537 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.878657 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.878722 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.878867 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.878912 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":120,"threads_started":1}
I20260809 16:29:30.879606 372 tablet_service.cc:1467] Processing CreateTablet for tablet 15da0e418e4347d680fa999b7c0a4d8e (DEFAULT_TABLE table=test-table [id=ed2945e8fa7c4a97ba61d506e3ef8411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:30.879925 372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15da0e418e4347d680fa999b7c0a4d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:30.882264 997 tablet_bootstrap.cc:492] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8: Bootstrap starting.
I20260809 16:29:30.883381 997 tablet_bootstrap.cc:654] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:30.883466 999 tablet_bootstrap.cc:492] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0: Bootstrap starting.
I20260809 16:29:30.883754 997 log.cc:824] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:30.883867 998 tablet_bootstrap.cc:492] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b: Bootstrap starting.
I20260809 16:29:30.884141 999 tablet_bootstrap.cc:654] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:30.884474 999 log.cc:824] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:30.884531 998 tablet_bootstrap.cc:654] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:30.884843 998 log.cc:824] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:30.885386 999 tablet_bootstrap.cc:492] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0: No bootstrap required, opened a new log
I20260809 16:29:30.885468 999 ts_tablet_manager.cc:1397] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:29:30.885628 998 tablet_bootstrap.cc:492] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b: No bootstrap required, opened a new log
I20260809 16:29:30.885692 998 ts_tablet_manager.cc:1397] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:29:30.887509 999 raft_consensus.cc:348] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } }
I20260809 16:29:30.887691 999 raft_consensus.cc:374] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:30.887728 999 raft_consensus.cc:729] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad7385e45cd54214b9672c92562fb3e0, State: Initialized, Role: FOLLOWER
I20260809 16:29:30.887655 998 raft_consensus.cc:348] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } }
I20260809 16:29:30.887785 998 raft_consensus.cc:374] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:30.887866 998 raft_consensus.cc:729] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 358bfead002847ddbddf9fd55abf839b, State: Initialized, Role: FOLLOWER
I20260809 16:29:30.887850 999 consensus_queue.cc:260] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } }
I20260809 16:29:30.887979 998 consensus_queue.cc:260] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } }
I20260809 16:29:30.888162 999 ts_tablet_manager.cc:1428] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 16:29:30.888221 476 heartbeater.cc:503] Master 127.30.23.190:38081 was elected leader, sending a full tablet report...
I20260809 16:29:30.888247 998 ts_tablet_manager.cc:1428] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 16:29:30.888360 717 heartbeater.cc:503] Master 127.30.23.190:38081 was elected leader, sending a full tablet report...
I20260809 16:29:30.889038 997 tablet_bootstrap.cc:492] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8: No bootstrap required, opened a new log
I20260809 16:29:30.889110 997 ts_tablet_manager.cc:1397] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 16:29:30.890769 997 raft_consensus.cc:348] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } }
I20260809 16:29:30.890933 997 raft_consensus.cc:374] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:30.890980 997 raft_consensus.cc:729] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69098f09c4e4ba391b4eea0bcb010d8, State: Initialized, Role: FOLLOWER
I20260809 16:29:30.891098 997 consensus_queue.cc:260] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } }
I20260809 16:29:30.891378 997 ts_tablet_manager.cc:1428] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:29:30.891486 959 heartbeater.cc:503] Master 127.30.23.190:38081 was elected leader, sending a full tablet report...
I20260809 16:29:30.895169 1005 raft_consensus.cc:482] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:29:30.895280 1005 raft_consensus.cc:504] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } }
I20260809 16:29:30.895567 1005 leader_election.cc:288] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e69098f09c4e4ba391b4eea0bcb010d8 (127.30.23.131:41253), ad7385e45cd54214b9672c92562fb3e0 (127.30.23.129:46083)
I20260809 16:29:30.900383 992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.896003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:54677 (local address 127.30.23.129:46083)
0809 16:29:30.896054 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.896056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.896068 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:29:30.896098 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.896099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.896109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.896171 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:30.896413 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.897175 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.899149 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.899791 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.900015 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.900030 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.900251 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.900300 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:29:30.900383 996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.896252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:50931 (local address 127.30.23.131:41253)
0809 16:29:30.896312 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.896314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.896321 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:29:30.896342 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.896343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.896351 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.896412 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:30.896597 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.897264 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.899150 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.899804 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.900015 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.900030 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.900251 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.900299 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:29:30.900400 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.895789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:46083 (local address 127.30.23.130:54677)
0809 16:29:30.895884 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:30.895905 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:30.895968 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:30.896210 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:30.896212 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:30.896214 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:30.896321 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.896322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.897440 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.897441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.899015 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.899019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.900024 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.900025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.900160 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.900165 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:29:30.900233 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 16:29:30.900297 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":74}
I20260809 16:29:30.900406 1008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.895789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:41253 (local address 127.30.23.130:50931)
0809 16:29:30.896050 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:30.896061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:30.896122 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:30.896448 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:30.896449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:30.896451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:30.896524 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.896526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.897440 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.897441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.899015 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:30.899019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.900024 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.900025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:30.900160 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.900165 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:29:30.900233 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 16:29:30.900294 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":146,"thread_start_us":167,"threads_started":1}
I20260809 16:29:30.900861 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15da0e418e4347d680fa999b7c0a4d8e" candidate_uuid: "358bfead002847ddbddf9fd55abf839b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad7385e45cd54214b9672c92562fb3e0" is_pre_election: true
I20260809 16:29:30.901048 405 raft_consensus.cc:2454] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 358bfead002847ddbddf9fd55abf839b in term 0.
I20260809 16:29:30.901294 521 leader_election.cc:302] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 358bfead002847ddbddf9fd55abf839b, ad7385e45cd54214b9672c92562fb3e0; no voters:
I20260809 16:29:30.901294 878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15da0e418e4347d680fa999b7c0a4d8e" candidate_uuid: "358bfead002847ddbddf9fd55abf839b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" is_pre_election: true
I20260809 16:29:30.901464 878 raft_consensus.cc:2454] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 358bfead002847ddbddf9fd55abf839b in term 0.
I20260809 16:29:30.901487 1005 raft_consensus.cc:2781] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:29:30.901724 1005 raft_consensus.cc:482] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:29:30.901777 1005 raft_consensus.cc:3037] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Advancing to term 1
W20260809 16:29:30.902402 719 tablet.cc:2369] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:29:30.902794 1005 raft_consensus.cc:504] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } }
I20260809 16:29:30.903025 1005 leader_election.cc:288] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [CANDIDATE]: Term 1 election: Requested vote from peers e69098f09c4e4ba391b4eea0bcb010d8 (127.30.23.131:41253), ad7385e45cd54214b9672c92562fb3e0 (127.30.23.129:46083)
I20260809 16:29:30.903132 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15da0e418e4347d680fa999b7c0a4d8e" candidate_uuid: "358bfead002847ddbddf9fd55abf839b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad7385e45cd54214b9672c92562fb3e0"
I20260809 16:29:30.903211 405 raft_consensus.cc:3037] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:30.903198 878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15da0e418e4347d680fa999b7c0a4d8e" candidate_uuid: "358bfead002847ddbddf9fd55abf839b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69098f09c4e4ba391b4eea0bcb010d8"
I20260809 16:29:30.903465 878 raft_consensus.cc:3037] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:30.903904 405 raft_consensus.cc:2454] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 358bfead002847ddbddf9fd55abf839b in term 1.
I20260809 16:29:30.904134 521 leader_election.cc:302] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 358bfead002847ddbddf9fd55abf839b, ad7385e45cd54214b9672c92562fb3e0; no voters:
I20260809 16:29:30.904277 1005 raft_consensus.cc:2781] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:30.904502 1005 raft_consensus.cc:686] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [term 1 LEADER]: Becoming Leader. State: Replica: 358bfead002847ddbddf9fd55abf839b, State: Running, Role: LEADER
I20260809 16:29:30.904605 1005 consensus_queue.cc:237] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } }
I20260809 16:29:30.905525 878 raft_consensus.cc:2454] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 358bfead002847ddbddf9fd55abf839b in term 1.
I20260809 16:29:30.905498 32624 catalog_manager.cc:5495] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b reported cstate change: term changed from 0 to 1, leader changed from <none> to 358bfead002847ddbddf9fd55abf839b (127.30.23.130). New cstate: current_term: 1 leader_uuid: "358bfead002847ddbddf9fd55abf839b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } health_report { overall_health: UNKNOWN } } }
I20260809 16:29:30.915519 995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:30.912799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52400 (local address 127.30.23.130:42477)
0809 16:29:30.912868 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:30.912871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:30.912881 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:29:30.912999 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:30.913000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:30.913027 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:30.913102 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:30.913249 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.913821 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.914626 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:30.914840 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:30.914924 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:30.915125 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:30.915228 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:30.915319 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:30.915374 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 16:29:30.915398 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:29:30.917428 405 raft_consensus.cc:1261] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0 [term 1 FOLLOWER]: Refusing update from remote peer 358bfead002847ddbddf9fd55abf839b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:30.917407 878 raft_consensus.cc:1261] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8 [term 1 FOLLOWER]: Refusing update from remote peer 358bfead002847ddbddf9fd55abf839b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:30.917775 1005 consensus_queue.cc:1035] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:30.917943 1009 consensus_queue.cc:1035] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:30.920898 1015 mvcc.cc:205] Tried to move back new op lower bound from 7316656008874373120 to 7316656008825913344. Current Snapshot: MvccSnapshot[applied={T|T < 7316656008874373120}]
I20260809 16:29:30.921023 1018 mvcc.cc:205] Tried to move back new op lower bound from 7316656008874373120 to 7316656008825913344. Current Snapshot: MvccSnapshot[applied={T|T < 7316656008874373120}]
I20260809 16:29:30.924994 32624 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:30.925114 32624 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:30.926085 32624 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726: Sending DeleteTablet for 3 replicas of tablet 15da0e418e4347d680fa999b7c0a4d8e
I20260809 16:29:30.926430 849 tablet_service.cc:1514] Processing DeleteTablet for tablet 15da0e418e4347d680fa999b7c0a4d8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:30 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:48808
I20260809 16:29:30.926460 372 tablet_service.cc:1514] Processing DeleteTablet for tablet 15da0e418e4347d680fa999b7c0a4d8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:30 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:39556
I20260809 16:29:30.926694 600 tablet_service.cc:1514] Processing DeleteTablet for tablet 15da0e418e4347d680fa999b7c0a4d8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:30 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:52386
I20260809 16:29:30.926930 1034 tablet_replica.cc:330] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0: stopping tablet replica
I20260809 16:29:30.926932 1036 tablet_replica.cc:330] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b: stopping tablet replica
I20260809 16:29:30.927039 1036 raft_consensus.cc:2229] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:29:30.927040 1034 raft_consensus.cc:2229] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:30.926991 32624 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:29:30.927132 32624 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:29:30.927153 1036 raft_consensus.cc:2258] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:30.927158 1034 raft_consensus.cc:2258] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:30.927485 1032 tablet_replica.cc:330] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8: stopping tablet replica
I20260809 16:29:30.927583 1032 raft_consensus.cc:2229] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:30.927680 1032 raft_consensus.cc:2258] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:30.928061 1034 ts_tablet_manager.cc:1905] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:30.928632 372 tablet_service.cc:1467] Processing CreateTablet for tablet fb2f8e663c0e419e8c3b332c59d1e96b (DEFAULT_TABLE table=test-table [id=61b7e9354b5b4e739f9443bd2084ab97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:30.928747 372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb2f8e663c0e419e8c3b332c59d1e96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:30.928884 1036 ts_tablet_manager.cc:1905] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:30.929232 849 tablet_service.cc:1467] Processing CreateTablet for tablet fb2f8e663c0e419e8c3b332c59d1e96b (DEFAULT_TABLE table=test-table [id=61b7e9354b5b4e739f9443bd2084ab97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:30.929494 849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb2f8e663c0e419e8c3b332c59d1e96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:30.930143 600 tablet_service.cc:1467] Processing CreateTablet for tablet fb2f8e663c0e419e8c3b332c59d1e96b (DEFAULT_TABLE table=test-table [id=61b7e9354b5b4e739f9443bd2084ab97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:30.930255 600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb2f8e663c0e419e8c3b332c59d1e96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:30.930925 997 tablet_bootstrap.cc:492] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8: Bootstrap starting.
I20260809 16:29:30.931178 1036 ts_tablet_manager.cc:1918] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:29:30.931249 1036 log.cc:1196] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/15da0e418e4347d680fa999b7c0a4d8e
I20260809 16:29:30.931384 997 tablet_bootstrap.cc:654] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:30.931540 1036 ts_tablet_manager.cc:1939] T 15da0e418e4347d680fa999b7c0a4d8e P 358bfead002847ddbddf9fd55abf839b: Deleting consensus metadata
I20260809 16:29:30.931613 999 tablet_bootstrap.cc:492] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0: Bootstrap starting.
I20260809 16:29:30.932029 999 tablet_bootstrap.cc:654] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:30.932363 998 tablet_bootstrap.cc:492] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b: Bootstrap starting.
I20260809 16:29:30.932662 999 tablet_bootstrap.cc:492] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0: No bootstrap required, opened a new log
I20260809 16:29:30.932729 999 ts_tablet_manager.cc:1397] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:30.932765 998 tablet_bootstrap.cc:654] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:30.932909 999 raft_consensus.cc:348] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:30.932996 999 raft_consensus.cc:374] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:30.933025 999 raft_consensus.cc:729] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad7385e45cd54214b9672c92562fb3e0, State: Initialized, Role: FOLLOWER
I20260809 16:29:30.933070 999 consensus_queue.cc:260] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:30.933173 999 ts_tablet_manager.cc:1428] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:29:30.933183 32597 catalog_manager.cc:4841] TS 358bfead002847ddbddf9fd55abf839b (127.30.23.130:42477): tablet 15da0e418e4347d680fa999b7c0a4d8e (table test-table [id=ed2945e8fa7c4a97ba61d506e3ef8411]) successfully deleted
I20260809 16:29:30.934211 1032 ts_tablet_manager.cc:1905] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:30.934908 1034 ts_tablet_manager.cc:1918] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:29:30.934994 1034 log.cc:1196] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/wal/wals/15da0e418e4347d680fa999b7c0a4d8e
I20260809 16:29:30.935282 1034 ts_tablet_manager.cc:1939] T 15da0e418e4347d680fa999b7c0a4d8e P ad7385e45cd54214b9672c92562fb3e0: Deleting consensus metadata
I20260809 16:29:30.936057 1032 ts_tablet_manager.cc:1918] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:29:30.936090 32602 catalog_manager.cc:4841] TS ad7385e45cd54214b9672c92562fb3e0 (127.30.23.129:46083): tablet 15da0e418e4347d680fa999b7c0a4d8e (table test-table [id=ed2945e8fa7c4a97ba61d506e3ef8411]) successfully deleted
I20260809 16:29:30.936132 1032 log.cc:1196] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/15da0e418e4347d680fa999b7c0a4d8e
I20260809 16:29:30.936395 1032 ts_tablet_manager.cc:1939] T 15da0e418e4347d680fa999b7c0a4d8e P e69098f09c4e4ba391b4eea0bcb010d8: Deleting consensus metadata
I20260809 16:29:30.936791 32597 catalog_manager.cc:4841] TS e69098f09c4e4ba391b4eea0bcb010d8 (127.30.23.131:41253): tablet 15da0e418e4347d680fa999b7c0a4d8e (table test-table [id=ed2945e8fa7c4a97ba61d506e3ef8411]) successfully deleted
I20260809 16:29:30.937812 998 tablet_bootstrap.cc:492] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b: No bootstrap required, opened a new log
I20260809 16:29:30.937876 998 ts_tablet_manager.cc:1397] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260809 16:29:30.938102 997 tablet_bootstrap.cc:492] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8: No bootstrap required, opened a new log
I20260809 16:29:30.938146 997 ts_tablet_manager.cc:1397] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260809 16:29:30.938189 998 raft_consensus.cc:348] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:30.938269 998 raft_consensus.cc:374] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:30.938292 998 raft_consensus.cc:729] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 358bfead002847ddbddf9fd55abf839b, State: Initialized, Role: FOLLOWER
I20260809 16:29:30.938288 997 raft_consensus.cc:348] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:30.938352 997 raft_consensus.cc:374] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:30.938374 997 raft_consensus.cc:729] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69098f09c4e4ba391b4eea0bcb010d8, State: Initialized, Role: FOLLOWER
I20260809 16:29:30.938346 998 consensus_queue.cc:260] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:30.938418 997 consensus_queue.cc:260] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:30.938453 998 ts_tablet_manager.cc:1428] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:30.938522 997 ts_tablet_manager.cc:1428] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260809 16:29:30.964284 478 tablet.cc:2369] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:29:31.074400 962 tablet.cc:2369] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:29:31.198351 1004 raft_consensus.cc:482] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:29:31.198463 1004 raft_consensus.cc:504] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:31.198820 1004 leader_election.cc:288] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 358bfead002847ddbddf9fd55abf839b (127.30.23.130:42477), e69098f09c4e4ba391b4eea0bcb010d8 (127.30.23.131:41253)
I20260809 16:29:31.201948 1041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.199085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:42477 (local address 127.30.23.129:46075)
0809 16:29:31.199216 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:31.199228 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:31.199289 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:31.199476 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:31.199478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:31.199485 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:31.199562 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.199564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.200228 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.200229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.201047 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.201049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.201660 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.201661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.201740 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.201744 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:31.201816 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:29:31.201860 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":24,"thread_start_us":80,"threads_started":1}
I20260809 16:29:31.201931 995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.199198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:46075 (local address 127.30.23.130:42477)
0809 16:29:31.199284 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:31.199286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:31.199300 (+ 14us) server_negotiation.cc:407] Connection header received
0809 16:29:31.199343 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:31.199344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:31.199358 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:31.199443 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:31.199637 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.200132 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.201156 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.201580 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.201662 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.201671 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:31.201820 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 16:29:31.201860 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:29:31.202055 996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.199381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:46891 (local address 127.30.23.131:41253)
0809 16:29:31.199458 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:31.199460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:31.199476 (+ 16us) server_negotiation.cc:407] Connection header received
0809 16:29:31.199530 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:31.199531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:31.199547 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:31.199606 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:31.199774 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.200283 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.201277 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.201699 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.201781 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.201789 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:31.201941 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 16:29:31.201976 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 16:29:31.202066 1045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.199281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:41253 (local address 127.30.23.129:46891)
0809 16:29:31.199387 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:31.199420 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:31.199464 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:31.199624 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:31.199625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:31.199627 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:31.199702 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.199704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.200407 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.200408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.201191 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.201194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.201780 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.201781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.201866 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.201869 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:29:31.201936 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:29:31.201975 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":41,"threads_started":1}
I20260809 16:29:31.202327 878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb2f8e663c0e419e8c3b332c59d1e96b" candidate_uuid: "ad7385e45cd54214b9672c92562fb3e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" is_pre_election: true
I20260809 16:29:31.202373 633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb2f8e663c0e419e8c3b332c59d1e96b" candidate_uuid: "ad7385e45cd54214b9672c92562fb3e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "358bfead002847ddbddf9fd55abf839b" is_pre_election: true
I20260809 16:29:31.202450 878 raft_consensus.cc:2454] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad7385e45cd54214b9672c92562fb3e0 in term 0.
I20260809 16:29:31.202508 633 raft_consensus.cc:2454] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad7385e45cd54214b9672c92562fb3e0 in term 0.
I20260809 16:29:31.202644 32765 leader_election.cc:302] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad7385e45cd54214b9672c92562fb3e0, e69098f09c4e4ba391b4eea0bcb010d8; no voters:
I20260809 16:29:31.202804 1004 raft_consensus.cc:2781] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:29:31.202879 1004 raft_consensus.cc:482] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:29:31.202914 1004 raft_consensus.cc:3037] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:31.203603 1004 raft_consensus.cc:504] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:31.203763 1004 leader_election.cc:288] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [CANDIDATE]: Term 1 election: Requested vote from peers 358bfead002847ddbddf9fd55abf839b (127.30.23.130:42477), e69098f09c4e4ba391b4eea0bcb010d8 (127.30.23.131:41253)
I20260809 16:29:31.203959 878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb2f8e663c0e419e8c3b332c59d1e96b" candidate_uuid: "ad7385e45cd54214b9672c92562fb3e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69098f09c4e4ba391b4eea0bcb010d8"
I20260809 16:29:31.203965 633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb2f8e663c0e419e8c3b332c59d1e96b" candidate_uuid: "ad7385e45cd54214b9672c92562fb3e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "358bfead002847ddbddf9fd55abf839b"
I20260809 16:29:31.204041 878 raft_consensus.cc:3037] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:31.204041 633 raft_consensus.cc:3037] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:31.204803 878 raft_consensus.cc:2454] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad7385e45cd54214b9672c92562fb3e0 in term 1.
I20260809 16:29:31.205009 633 raft_consensus.cc:2454] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad7385e45cd54214b9672c92562fb3e0 in term 1.
I20260809 16:29:31.205029 32765 leader_election.cc:302] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad7385e45cd54214b9672c92562fb3e0, e69098f09c4e4ba391b4eea0bcb010d8; no voters:
I20260809 16:29:31.205156 1004 raft_consensus.cc:2781] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:31.205295 1004 raft_consensus.cc:686] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [term 1 LEADER]: Becoming Leader. State: Replica: ad7385e45cd54214b9672c92562fb3e0, State: Running, Role: LEADER
I20260809 16:29:31.205401 1004 consensus_queue.cc:237] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:31.206161 32625 catalog_manager.cc:5495] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad7385e45cd54214b9672c92562fb3e0 (127.30.23.129). New cstate: current_term: 1 leader_uuid: "ad7385e45cd54214b9672c92562fb3e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } health_report { overall_health: UNKNOWN } } }
Aug 09 16:29:31 dist-test-slave-j2vl krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292970, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:31.245849 1060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.238199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38081 (local address 127.0.0.1:33052)
0809 16:29:31.238382 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:31.238401 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:31.238471 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:31.238606 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:31.238607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:31.238918 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:31.238973 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.238975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.239688 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.239689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.240266 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.240268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.240718 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.240719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.240778 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.241332 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:31.241337 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:31.242848 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:31.244632 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:31.244640 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:31.244642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:31.244858 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:31.245145 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:31.245146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:31.245146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:31.245219 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:31.245534 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:31.245542 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:31.245589 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 16:29:31.245661 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:29:31.245764 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":118,"threads_started":1}
I20260809 16:29:31.245951 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.238264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.30.23.190:38081)
0809 16:29:31.238346 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:31.238348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:31.238476 (+ 128us) server_negotiation.cc:407] Connection header received
0809 16:29:31.238512 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:31.238513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:31.238522 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:31.238589 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:31.239068 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.239581 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.240422 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.240620 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.240677 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.242956 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:31.242957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:31.242957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:31.242964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:31.244537 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:31.244970 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:31.244971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:31.244971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:31.245041 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:31.245290 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:31.245291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:31.245291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:31.245464 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:31.245521 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:31.245690 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 16:29:31.245815 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":43}
I20260809 16:29:31.248111 32625 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:33052:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 16:29:31.248387 32625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:29:31.249997 372 tablet_service.cc:1467] Processing CreateTablet for tablet b844513407c14ce98c3e0a69692ada2a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1be6e5fe8d184b0e80273ab8fcf5cd6f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:29:31.250051 600 tablet_service.cc:1467] Processing CreateTablet for tablet b844513407c14ce98c3e0a69692ada2a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1be6e5fe8d184b0e80273ab8fcf5cd6f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:29:31.250051 849 tablet_service.cc:1467] Processing CreateTablet for tablet b844513407c14ce98c3e0a69692ada2a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1be6e5fe8d184b0e80273ab8fcf5cd6f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:29:31.250185 372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b844513407c14ce98c3e0a69692ada2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:31.250213 600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b844513407c14ce98c3e0a69692ada2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:31.250212 849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b844513407c14ce98c3e0a69692ada2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:31.252614 997 tablet_bootstrap.cc:492] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8: Bootstrap starting.
I20260809 16:29:31.252614 998 tablet_bootstrap.cc:492] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b: Bootstrap starting.
I20260809 16:29:31.252607 999 tablet_bootstrap.cc:492] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0: Bootstrap starting.
I20260809 16:29:31.253052 998 tablet_bootstrap.cc:654] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:31.253069 997 tablet_bootstrap.cc:654] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:31.253114 999 tablet_bootstrap.cc:654] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:31.253863 997 tablet_bootstrap.cc:492] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8: No bootstrap required, opened a new log
I20260809 16:29:31.253899 999 tablet_bootstrap.cc:492] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0: No bootstrap required, opened a new log
I20260809 16:29:31.253923 997 ts_tablet_manager.cc:1397] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:31.253950 999 ts_tablet_manager.cc:1397] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:31.253954 998 tablet_bootstrap.cc:492] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b: No bootstrap required, opened a new log
I20260809 16:29:31.254000 998 ts_tablet_manager.cc:1397] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:31.254112 998 raft_consensus.cc:348] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:31.254097 999 raft_consensus.cc:348] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:31.254089 997 raft_consensus.cc:348] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:31.254163 998 raft_consensus.cc:374] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:31.254164 999 raft_consensus.cc:374] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:31.254175 998 raft_consensus.cc:729] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 358bfead002847ddbddf9fd55abf839b, State: Initialized, Role: FOLLOWER
I20260809 16:29:31.254174 997 raft_consensus.cc:374] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:31.254191 999 raft_consensus.cc:729] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad7385e45cd54214b9672c92562fb3e0, State: Initialized, Role: FOLLOWER
I20260809 16:29:31.254201 997 raft_consensus.cc:729] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69098f09c4e4ba391b4eea0bcb010d8, State: Initialized, Role: FOLLOWER
I20260809 16:29:31.254236 998 consensus_queue.cc:260] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:31.254240 999 consensus_queue.cc:260] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:31.254247 997 consensus_queue.cc:260] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:31.254338 998 ts_tablet_manager.cc:1428] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:31.254338 997 ts_tablet_manager.cc:1428] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:29:31.254349 999 ts_tablet_manager.cc:1428] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:31.254369 1025 tablet_replica.cc:439] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } }
I20260809 16:29:31.254443 1025 tablet_replica.cc:442] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:31.254567 1004 tablet_replica.cc:439] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } }
I20260809 16:29:31.254663 1004 tablet_replica.cc:442] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:31.254813 1007 tablet_replica.cc:439] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } }
I20260809 16:29:31.254921 1007 tablet_replica.cc:442] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:31.411284 1007 raft_consensus.cc:482] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:29:31.411433 1007 raft_consensus.cc:504] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:31.411872 1007 leader_election.cc:288] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad7385e45cd54214b9672c92562fb3e0 (127.30.23.129:46083), 358bfead002847ddbddf9fd55abf839b (127.30.23.130:42477)
I20260809 16:29:31.415961 995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.412243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:35441 (local address 127.30.23.130:42477)
0809 16:29:31.412290 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:31.412292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:31.412303 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:29:31.412390 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:31.412391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:31.412401 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:31.412463 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:31.412635 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.413462 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.414983 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.415537 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.415633 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.415644 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:31.415836 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 16:29:31.415894 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:29:31.415974 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.412101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:42477 (local address 127.30.23.131:35441)
0809 16:29:31.412239 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:31.412252 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:31.412312 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:31.412477 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:31.412478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:31.412482 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:31.412571 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.412573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.413536 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.413537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.414826 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.414830 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.415648 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.415649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.415737 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.415743 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:29:31.415820 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:29:31.415858 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":3,"thread_start_us":67,"threads_started":1}
I20260809 16:29:31.416105 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.412170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:34857 (local address 127.30.23.129:46083)
0809 16:29:31.412278 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:31.412280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:31.412324 (+ 44us) server_negotiation.cc:407] Connection header received
0809 16:29:31.412461 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:31.412463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:31.412474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:31.412546 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:31.412756 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.413513 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.414975 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.415649 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.415714 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.415722 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:31.415914 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 16:29:31.415957 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260809 16:29:31.416245 633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b844513407c14ce98c3e0a69692ada2a" candidate_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "358bfead002847ddbddf9fd55abf839b" is_pre_election: true
I20260809 16:29:31.416329 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.412081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:46083 (local address 127.30.23.131:34857)
0809 16:29:31.412234 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:31.412247 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:31.412367 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:31.412553 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:31.412554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:31.412570 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:31.412670 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.412672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.413583 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.413584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.414844 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.414846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.415714 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.415715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.415827 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.415831 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:31.415889 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:29:31.415944 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":61,"thread_start_us":81,"threads_started":1}
I20260809 16:29:31.416369 633 raft_consensus.cc:2454] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e69098f09c4e4ba391b4eea0bcb010d8 in term 0.
I20260809 16:29:31.416579 764 leader_election.cc:302] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 358bfead002847ddbddf9fd55abf839b, e69098f09c4e4ba391b4eea0bcb010d8; no voters:
I20260809 16:29:31.416594 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b844513407c14ce98c3e0a69692ada2a" candidate_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad7385e45cd54214b9672c92562fb3e0" is_pre_election: true
I20260809 16:29:31.416700 405 raft_consensus.cc:2454] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e69098f09c4e4ba391b4eea0bcb010d8 in term 0.
I20260809 16:29:31.416718 1007 raft_consensus.cc:2781] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:29:31.416759 1007 raft_consensus.cc:482] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:29:31.416790 1007 raft_consensus.cc:3037] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:31.417557 1007 raft_consensus.cc:504] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:31.417697 1007 leader_election.cc:288] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [CANDIDATE]: Term 1 election: Requested vote from peers ad7385e45cd54214b9672c92562fb3e0 (127.30.23.129:46083), 358bfead002847ddbddf9fd55abf839b (127.30.23.130:42477)
I20260809 16:29:31.417881 633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b844513407c14ce98c3e0a69692ada2a" candidate_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "358bfead002847ddbddf9fd55abf839b"
I20260809 16:29:31.417958 633 raft_consensus.cc:3037] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:31.418578 633 raft_consensus.cc:2454] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69098f09c4e4ba391b4eea0bcb010d8 in term 1.
I20260809 16:29:31.418671 405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b844513407c14ce98c3e0a69692ada2a" candidate_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad7385e45cd54214b9672c92562fb3e0"
I20260809 16:29:31.418774 405 raft_consensus.cc:3037] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:31.418756 764 leader_election.cc:302] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 358bfead002847ddbddf9fd55abf839b, e69098f09c4e4ba391b4eea0bcb010d8; no voters:
I20260809 16:29:31.418889 1007 raft_consensus.cc:2781] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:31.419014 1007 raft_consensus.cc:686] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [term 1 LEADER]: Becoming Leader. State: Replica: e69098f09c4e4ba391b4eea0bcb010d8, State: Running, Role: LEADER
I20260809 16:29:31.419085 1007 consensus_queue.cc:237] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } }
I20260809 16:29:31.419454 1071 tablet_replica.cc:439] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8: TxnStatusTablet state changed. Reason: New leader e69098f09c4e4ba391b4eea0bcb010d8. Latest consensus state: current_term: 1 leader_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } }
I20260809 16:29:31.419596 1071 tablet_replica.cc:442] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:29:31.419608 405 raft_consensus.cc:2454] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69098f09c4e4ba391b4eea0bcb010d8 in term 1.
I20260809 16:29:31.419849 1075 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 16:29:31.419951 1075 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 16:29:31.419843 32625 catalog_manager.cc:5495] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e69098f09c4e4ba391b4eea0bcb010d8 (127.30.23.131). New cstate: current_term: 1 leader_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } health_report { overall_health: HEALTHY } } }
I20260809 16:29:31.438354 1060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.435107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:41253 (local address 127.0.0.1:48822)
0809 16:29:31.435186 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:31.435199 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:31.435254 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:31.435624 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:31.435625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:31.435627 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:29:31.435680 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.435682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.436636 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.436637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.437219 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.437221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.437788 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.437790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.437891 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.437907 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:29:31.438202 (+ 295us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:29:31.438210 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 16:29:31.438247 (+ 37us) client_negotiation.cc:240] Negotiation successful
0809 16:29:31.438295 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 16:29:31.438367 996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.435360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48822 (local address 127.30.23.131:41253)
0809 16:29:31.435431 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:31.435433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:31.435443 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:29:31.435476 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:31.435477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:31.435499 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:31.435576 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:31.435763 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.436539 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.437300 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.437624 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.437699 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.438006 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:31.438133 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:31.438210 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:31.438262 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 16:29:31.438284 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:29:31.439750 405 raft_consensus.cc:1261] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0 [term 1 FOLLOWER]: Refusing update from remote peer e69098f09c4e4ba391b4eea0bcb010d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:31.439790 633 raft_consensus.cc:1261] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b [term 1 FOLLOWER]: Refusing update from remote peer e69098f09c4e4ba391b4eea0bcb010d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:31.440138 1071 consensus_queue.cc:1035] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:31.440284 1007 consensus_queue.cc:1035] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:31.440982 1025 tablet_replica.cc:439] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b: TxnStatusTablet state changed. Reason: New leader e69098f09c4e4ba391b4eea0bcb010d8. Latest consensus state: current_term: 1 leader_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } }
I20260809 16:29:31.440994 1004 tablet_replica.cc:439] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0: TxnStatusTablet state changed. Reason: New leader e69098f09c4e4ba391b4eea0bcb010d8. Latest consensus state: current_term: 1 leader_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } }
I20260809 16:29:31.441085 1004 tablet_replica.cc:442] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:31.441087 1025 tablet_replica.cc:442] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:31.442142 1025 tablet_replica.cc:439] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } }
I20260809 16:29:31.442250 1025 tablet_replica.cc:442] T b844513407c14ce98c3e0a69692ada2a P 358bfead002847ddbddf9fd55abf839b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:31.441953 1071 tablet_replica.cc:439] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } }
I20260809 16:29:31.442392 1071 tablet_replica.cc:442] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:29:31.442408 1007 tablet_replica.cc:439] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } }
I20260809 16:29:31.442562 1007 tablet_replica.cc:442] T b844513407c14ce98c3e0a69692ada2a P e69098f09c4e4ba391b4eea0bcb010d8: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:29:31.442639 1004 tablet_replica.cc:439] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7385e45cd54214b9672c92562fb3e0" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46083 } } peers { permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 } } peers { permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 } } }
I20260809 16:29:31.442730 1004 tablet_replica.cc:442] T b844513407c14ce98c3e0a69692ada2a P ad7385e45cd54214b9672c92562fb3e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:31.443552 1087 mvcc.cc:205] Tried to move back new op lower bound from 7316656011014721536 to 7316656010933190656. Current Snapshot: MvccSnapshot[applied={T|T < 7316656011014721536}]
I20260809 16:29:31.447876 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.445110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39562 (local address 127.30.23.129:46083)
0809 16:29:31.445155 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:31.445157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:31.445199 (+ 42us) server_negotiation.cc:407] Connection header received
0809 16:29:31.445289 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:31.445290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:31.445308 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:31.445447 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:31.445606 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.446146 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.446970 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.447204 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.447289 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.447487 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:31.447598 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:31.447691 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:31.447748 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 16:29:31.447776 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:29:31.453454 996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.450683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:46733 (local address 127.30.23.131:41253)
0809 16:29:31.450732 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:31.450734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:31.450882 (+ 148us) server_negotiation.cc:407] Connection header received
0809 16:29:31.450936 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:31.450937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:31.450959 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:31.451026 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:31.451135 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.451852 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.452617 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.452911 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.452982 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.453177 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:31.453212 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:31.453266 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:31.453354 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 16:29:31.453374 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:29:31.453531 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.450618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:41253 (local address 127.30.23.129:46733)
0809 16:29:31.450800 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:31.450811 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:31.450892 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:31.451026 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:31.451027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:31.451028 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:29:31.451079 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.451080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.451945 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.451946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.452531 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.452532 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.453011 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.453012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.453069 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.453086 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:29:31.453280 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:29:31.453282 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:29:31.453324 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 16:29:31.453429 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260809 16:29:31.456159 878 raft_consensus.cc:1261] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8 [term 1 FOLLOWER]: Refusing update from remote peer ad7385e45cd54214b9672c92562fb3e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:31.456174 633 raft_consensus.cc:1261] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b [term 1 FOLLOWER]: Refusing update from remote peer ad7385e45cd54214b9672c92562fb3e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:31.456472 1048 consensus_queue.cc:1035] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69098f09c4e4ba391b4eea0bcb010d8" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:31.456635 1004 consensus_queue.cc:1035] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "358bfead002847ddbddf9fd55abf839b" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:31.467847 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.464709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:34489 (local address 127.30.23.129:46083)
0809 16:29:31.464749 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:31.464751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:31.464759 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:29:31.464782 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:31.464783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:31.464794 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:31.464863 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:31.465027 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.465648 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.467012 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.467276 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.467319 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.467580 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:31.467620 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:31.467685 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:31.467753 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 16:29:31.467778 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:29:31.467929 1106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.464066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:46083 (local address 127.30.23.131:34489)
0809 16:29:31.464666 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:31.464678 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:31.464733 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:31.464868 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:31.464869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:31.464871 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:29:31.464943 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.464945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.466012 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.466013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.466913 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:31.466916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.467404 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.467405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:31.467490 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.467506 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:29:31.467700 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:29:31.467703 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:29:31.467749 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:29:31.467802 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":525,"threads_started":1}
I20260809 16:29:31.482784 32625 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:31.482923 32625 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:31.483677 32625 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b59439fa4ab4e12bd2f4c6d1f4f8726: Sending DeleteTablet for 3 replicas of tablet fb2f8e663c0e419e8c3b332c59d1e96b
I20260809 16:29:31.483940 372 tablet_service.cc:1514] Processing DeleteTablet for tablet fb2f8e663c0e419e8c3b332c59d1e96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:31 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:39556
I20260809 16:29:31.484140 600 tablet_service.cc:1514] Processing DeleteTablet for tablet fb2f8e663c0e419e8c3b332c59d1e96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:31 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:52386
I20260809 16:29:31.484257 1113 tablet_replica.cc:330] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0: stopping tablet replica
I20260809 16:29:31.484314 1113 raft_consensus.cc:2229] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:29:31.484434 1113 raft_consensus.cc:2258] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:31.484783 1113 ts_tablet_manager.cc:1905] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:31.484966 849 tablet_service.cc:1514] Processing DeleteTablet for tablet fb2f8e663c0e419e8c3b332c59d1e96b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:31 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:48808
I20260809 16:29:31.485184 1115 tablet_replica.cc:330] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b: stopping tablet replica
I20260809 16:29:31.485293 1115 raft_consensus.cc:2229] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:31.485425 1115 raft_consensus.cc:2258] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:31.485728 1115 ts_tablet_manager.cc:1905] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:31.487056 1113 ts_tablet_manager.cc:1918] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:29:31.487128 1113 log.cc:1196] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-0/wal/wals/fb2f8e663c0e419e8c3b332c59d1e96b
I20260809 16:29:31.487525 1113 ts_tablet_manager.cc:1939] T fb2f8e663c0e419e8c3b332c59d1e96b P ad7385e45cd54214b9672c92562fb3e0: Deleting consensus metadata
I20260809 16:29:31.487888 32602 catalog_manager.cc:4841] TS ad7385e45cd54214b9672c92562fb3e0 (127.30.23.129:46083): tablet fb2f8e663c0e419e8c3b332c59d1e96b (table test-table [id=61b7e9354b5b4e739f9443bd2084ab97]) successfully deleted
I20260809 16:29:31.488296 1115 ts_tablet_manager.cc:1918] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:29:31.488363 1115 log.cc:1196] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/fb2f8e663c0e419e8c3b332c59d1e96b
I20260809 16:29:31.488596 1115 ts_tablet_manager.cc:1939] T fb2f8e663c0e419e8c3b332c59d1e96b P 358bfead002847ddbddf9fd55abf839b: Deleting consensus metadata
I20260809 16:29:31.489180 32597 catalog_manager.cc:4841] TS 358bfead002847ddbddf9fd55abf839b (127.30.23.130:42477): tablet fb2f8e663c0e419e8c3b332c59d1e96b (table test-table [id=61b7e9354b5b4e739f9443bd2084ab97]) successfully deleted
I20260809 16:29:31.489467 1121 tablet_replica.cc:330] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8: stopping tablet replica
I20260809 16:29:31.489543 1121 raft_consensus.cc:2229] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:31.489593 1121 raft_consensus.cc:2258] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:31.489864 1121 ts_tablet_manager.cc:1905] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:31.491433 1121 ts_tablet_manager.cc:1918] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:29:31.491498 1121 log.cc:1196] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/fb2f8e663c0e419e8c3b332c59d1e96b
I20260809 16:29:31.491724 1121 ts_tablet_manager.cc:1939] T fb2f8e663c0e419e8c3b332c59d1e96b P e69098f09c4e4ba391b4eea0bcb010d8: Deleting consensus metadata
I20260809 16:29:31.492115 32597 catalog_manager.cc:4841] TS e69098f09c4e4ba391b4eea0bcb010d8 (127.30.23.131:41253): tablet fb2f8e663c0e419e8c3b332c59d1e96b (table test-table [id=61b7e9354b5b4e739f9443bd2084ab97]) successfully deleted
Aug 09 16:29:31 dist-test-slave-j2vl krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292970, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.23.129@KRBTEST.COM
I20260809 16:29:31.503657 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.487444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39578 (local address 127.30.23.129:46083)
0809 16:29:31.487488 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:31.487490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:31.488866 (+ 1376us) server_negotiation.cc:407] Connection header received
0809 16:29:31.488925 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:31.488926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:31.488938 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:31.489002 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:31.490095 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.490843 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.492171 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.492483 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.492532 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.499919 (+ 7387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:31.499929 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:31.499930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:31.499940 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:31.502181 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:31.502633 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:31.502637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:31.502637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:31.502704 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:31.502947 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:31.502948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:31.502948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:31.503136 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:31.503281 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:31.503420 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 16:29:31.503513 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 16:29:31.503961 447 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39578
I20260809 16:29:31.513051 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:31.504988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33066 (local address 127.30.23.190:38081)
0809 16:29:31.505036 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:31.505038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:31.505410 (+ 372us) server_negotiation.cc:407] Connection header received
0809 16:29:31.505531 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:31.505533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:31.505558 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:31.505633 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:31.506346 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.507130 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.508123 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:31.508443 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:31.508516 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:31.510051 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:31.510053 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:31.510053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:31.510061 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:31.511773 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:31.512184 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:31.512185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:31.512186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:31.512247 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:31.512476 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:31.512477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:31.512477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:31.512612 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:31.512711 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:31.512871 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 16:29:31.512955 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 16:29:31.513303 32625 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33066
I20260809 16:29:31.514045 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 32740
I20260809 16:29:31.545199 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 491
I20260809 16:29:31.558419 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 731
I20260809 16:29:31.584590 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 32574
2026-08-09T16:29:31Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2937 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:29:31 dist-test-slave-j2vl krb5kdc[1140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:29:31 dist-test-slave-j2vl krb5kdc[1140](info): set up 2 sockets
Aug 09 16:29:31 dist-test-slave-j2vl krb5kdc[1140](info): commencing operation
krb5kdc: starting...
W20260809 16:29:32.895943 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:29:32 dist-test-slave-j2vl krb5kdc[1140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:29:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:29:33.028891 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:34131
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:34131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:33.112475 1164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:33.112759 1164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:33.112802 1164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:33.114012 1164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:33.114084 1164 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 16:29:33.114109 1164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:33.114142 1164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:33.114169 1164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:33.114194 1164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:33.115749 1164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:34131
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:34131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:33.116040 1164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:33.116281 1164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:33.121603 1177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:33.121646 1173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:33.121619 1172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:33.121984 1164 server_base.cc:1034] running on GCE node
I20260809 16:29:33.122289 1164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:33.122603 1164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:33.123811 1164 hybrid_clock.cc:648] HybridClock initialized: now 1786292973123794 us; error 43 us; skew 500 ppm
Aug 09 16:29:33 dist-test-slave-j2vl krb5kdc[1140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292973, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:33.129825 1164 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:29:33.130357 1164 webserver.cc:460] Webserver started at http://127.30.23.190:34423/ using document root <none> and password file <none>
I20260809 16:29:33.130599 1164 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:33.130669 1164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:33.130817 1164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:33.131944 1164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "25f744ade59a44e7be47378205a599bb"
format_stamp: "Formatted at 2026-08-09 16:29:33 on dist-test-slave-j2vl"
server_key: "d9c3a98fce0a919c426c45c000078922"
server_key_iv: "831d5513a74958f0bc4ef6685743f649"
server_key_version: "encryptionkey@0"
I20260809 16:29:33.132331 1164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "25f744ade59a44e7be47378205a599bb"
format_stamp: "Formatted at 2026-08-09 16:29:33 on dist-test-slave-j2vl"
server_key: "d9c3a98fce0a919c426c45c000078922"
server_key_iv: "831d5513a74958f0bc4ef6685743f649"
server_key_version: "encryptionkey@0"
I20260809 16:29:33.133870 1164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:29:33.134719 1191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:33.135027 1164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:33.135136 1164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "25f744ade59a44e7be47378205a599bb"
format_stamp: "Formatted at 2026-08-09 16:29:33 on dist-test-slave-j2vl"
server_key: "d9c3a98fce0a919c426c45c000078922"
server_key_iv: "831d5513a74958f0bc4ef6685743f649"
server_key_version: "encryptionkey@0"
I20260809 16:29:33.135215 1164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:33.142163 1164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:33.143970 1164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:33.144138 1164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:33.149729 1164 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:34131
I20260809 16:29:33.149782 1304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:34131 every 8 connection(s)
I20260809 16:29:33.150142 1164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:33.150765 1306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:33.153071 1306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb: Bootstrap starting.
I20260809 16:29:33.153739 1306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:33.154034 1306 log.cc:824] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:33.154771 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 1164
I20260809 16:29:33.154875 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:33.154878 1306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb: No bootstrap required, opened a new log
I20260809 16:29:33.155074 30814 external_mini_cluster.cc:1428] Setting key f3e983a5e420bbb668466fea2a2da308
I20260809 16:29:33.157198 1306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f744ade59a44e7be47378205a599bb" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34131 } }
I20260809 16:29:33.157330 1306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:33.157449 1306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f744ade59a44e7be47378205a599bb, State: Initialized, Role: FOLLOWER
I20260809 16:29:33.157598 1306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f744ade59a44e7be47378205a599bb" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34131 } }
I20260809 16:29:33.157686 1306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:33.157730 1306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:33.157778 1306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:33.158325 1306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f744ade59a44e7be47378205a599bb" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34131 } }
I20260809 16:29:33.158465 1306 leader_election.cc:302] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25f744ade59a44e7be47378205a599bb; no voters:
I20260809 16:29:33.158620 1306 leader_election.cc:288] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:33.158697 1314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:33.158847 1314 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [term 1 LEADER]: Becoming Leader. State: Replica: 25f744ade59a44e7be47378205a599bb, State: Running, Role: LEADER
I20260809 16:29:33.158870 1306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:33.158939 1314 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f744ade59a44e7be47378205a599bb" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34131 } }
I20260809 16:29:33.159262 1315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25f744ade59a44e7be47378205a599bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f744ade59a44e7be47378205a599bb" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34131 } } }
I20260809 16:29:33.159354 1315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:33.159602 1317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25f744ade59a44e7be47378205a599bb. Latest consensus state: current_term: 1 leader_uuid: "25f744ade59a44e7be47378205a599bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f744ade59a44e7be47378205a599bb" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34131 } } }
I20260809 16:29:33.159724 1317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:33.159927 1321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:33.160547 1321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:33.162402 1321 catalog_manager.cc:1347] Generated new cluster ID: 3ce25e75c9054dc6ab3e42f8a8e50289
I20260809 16:29:33.162459 1321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:29:33 dist-test-slave-j2vl krb5kdc[1140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:33.167929 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.155621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59292 (local address 127.30.23.190:34131)
0809 16:29:33.155821 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.155829 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.155855 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:29:33.157212 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.157221 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.157291 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.157504 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:33.158427 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.159990 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.161222 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.161865 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.161985 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.164514 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:33.164518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:33.164518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:33.164532 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:33.166329 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.166734 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.166737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.166737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.166848 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.167073 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.167074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.167075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.167336 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:33.167500 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.167607 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.167767 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":82,"threads_started":1}
I20260809 16:29:33.169808 1321 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:33.170291 1321 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:33.172960 1321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb: Generated new TSK 0
I20260809 16:29:33.173130 1321 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:29:33.205437 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34131
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:33.301086 1341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:33.301339 1341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:33.301400 1341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:33.302624 1341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:33.302701 1341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:33.302757 1341 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 16:29:33.302801 1341 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:29:33.302842 1341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:29:33.304371 1341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.23.190:34131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:33.304646 1341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:33.304893 1341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:33.311237 1350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:33.311437 1341 server_base.cc:1034] running on GCE node
W20260809 16:29:33.311581 1349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:33.311702 1353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:33.311940 1341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:33.312211 1341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:33.313385 1341 hybrid_clock.cc:648] HybridClock initialized: now 1786292973313367 us; error 38 us; skew 500 ppm
Aug 09 16:29:33 dist-test-slave-j2vl krb5kdc[1140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292973, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:33.316483 1341 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:29:33.316977 1341 webserver.cc:460] Webserver started at http://127.30.23.129:44347/ using document root <none> and password file <none>
I20260809 16:29:33.317214 1341 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:33.317304 1341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:33.317497 1341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:33.318665 1341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "a2a6b20d23404b649a1a94d7d47a01c1"
format_stamp: "Formatted at 2026-08-09 16:29:33 on dist-test-slave-j2vl"
server_key: "f08143dc5619259d259a16e9e7ab8794"
server_key_iv: "09eda54d3f59922e5f193f4b7f451ce0"
server_key_version: "encryptionkey@0"
I20260809 16:29:33.319048 1341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "a2a6b20d23404b649a1a94d7d47a01c1"
format_stamp: "Formatted at 2026-08-09 16:29:33 on dist-test-slave-j2vl"
server_key: "f08143dc5619259d259a16e9e7ab8794"
server_key_iv: "09eda54d3f59922e5f193f4b7f451ce0"
server_key_version: "encryptionkey@0"
I20260809 16:29:33.320536 1341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:29:33.321483 1369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:33.321713 1341 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:33.321830 1341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "a2a6b20d23404b649a1a94d7d47a01c1"
format_stamp: "Formatted at 2026-08-09 16:29:33 on dist-test-slave-j2vl"
server_key: "f08143dc5619259d259a16e9e7ab8794"
server_key_iv: "09eda54d3f59922e5f193f4b7f451ce0"
server_key_version: "encryptionkey@0"
I20260809 16:29:33.321924 1341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:33.332384 1341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:33.335027 1341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:33.335207 1341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:33.335778 1341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:33.335844 1341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:33.335894 1341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:33.335924 1341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:33.343089 1341 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:45619
I20260809 16:29:33.343176 1547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:45619 every 8 connection(s)
I20260809 16:29:33.343508 1341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:29:33 dist-test-slave-j2vl krb5kdc[1140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292973, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:33.347121 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.336400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:43749 (local address 127.30.23.190:34131)
0809 16:29:33.336455 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.336458 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.336694 (+ 236us) server_negotiation.cc:407] Connection header received
0809 16:29:33.337255 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.337257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.337277 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.337377 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:33.338380 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.339252 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.340966 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.341271 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.341414 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.344148 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:33.344150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:33.344150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:33.344160 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:33.345607 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.346050 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.346051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.346052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.346125 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.346403 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.346404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.346405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.346538 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:33.346617 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.346929 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.347021 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:29:33.347334 1384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.336398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34131 (local address 127.30.23.129:43749)
0809 16:29:33.336573 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.336610 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.337142 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.337503 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.337508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.337949 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:33.338281 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.338287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.339349 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.339350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.340875 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.340879 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.341407 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.341408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.341584 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.342274 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:33.342285 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:33.344041 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:33.345705 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.345712 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.345715 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.345963 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.346207 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.346207 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.346208 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.346325 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.346620 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:33.346629 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:33.346783 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.346903 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.347079 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":105,"threads_started":1}
I20260809 16:29:33.347939 1551 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34131
I20260809 16:29:33.348034 1551 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:33.348232 1551 heartbeater.cc:511] Master 127.30.23.190:34131 requested a full tablet report, sending...
I20260809 16:29:33.348841 1219 ts_manager.cc:194] Registered new tserver with Master: a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129:45619)
I20260809 16:29:33.349776 1219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:43749
I20260809 16:29:33.352334 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 1341
I20260809 16:29:33.352448 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:29:33.352640 30814 external_mini_cluster.cc:1428] Setting key daab69f67c330fb70fb03cc3cd81adbe
I20260809 16:29:33.355338 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.349233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:38579 (local address 127.30.23.190:34131)
0809 16:29:33.349282 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.349284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.349436 (+ 152us) server_negotiation.cc:407] Connection header received
0809 16:29:33.349516 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.349517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.349527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.349583 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:33.349909 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.350483 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.351177 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.351357 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.351424 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.352439 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:33.352441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:33.352442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:33.352449 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:33.353982 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.354409 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.354410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.354410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.354468 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.354697 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.354698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.354698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.354854 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:33.354976 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.355119 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.355219 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:29:33.355387 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.349170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34131 (local address 127.30.23.129:38579)
0809 16:29:33.349376 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.349388 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.349459 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.349595 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.349595 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.349748 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:33.349802 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.349803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.350586 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.350587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.351091 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.351093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.351451 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.351452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.351524 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.351929 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:33.351933 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:33.352297 (+ 364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:33.354067 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.354068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.354069 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.354319 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.354544 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.354545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.354546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.354619 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.354964 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:33.354965 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:33.355020 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.355101 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.355228 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:29:33.381028 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34131
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:33.464781 1562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:33.464990 1562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:33.465015 1562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:33.466112 1562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:33.466146 1562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:33.466166 1562 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 16:29:33.466179 1562 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:29:33.466197 1562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:29:33.467556 1562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.23.190:34131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:33.467741 1562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:33.467926 1562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:33.473136 1579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:33.473130 1575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:33.473130 1577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:33.473371 1562 server_base.cc:1034] running on GCE node
I20260809 16:29:33.473577 1562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:33.473873 1562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:33.475035 1562 hybrid_clock.cc:648] HybridClock initialized: now 1786292973475026 us; error 40 us; skew 500 ppm
Aug 09 16:29:33 dist-test-slave-j2vl krb5kdc[1140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292973, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:33.477898 1562 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:29:33.478408 1562 webserver.cc:460] Webserver started at http://127.30.23.130:40987/ using document root <none> and password file <none>
I20260809 16:29:33.478673 1562 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:33.478744 1562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:33.478912 1562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:33.480010 1562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "e70fb780c0f243c1bcb8099b4492a7d5"
format_stamp: "Formatted at 2026-08-09 16:29:33 on dist-test-slave-j2vl"
server_key: "c17b34cec677d5f434c3ad6aec63285f"
server_key_iv: "cdcce27ee021983be7fa2fd1a43538e5"
server_key_version: "encryptionkey@0"
I20260809 16:29:33.480377 1562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "e70fb780c0f243c1bcb8099b4492a7d5"
format_stamp: "Formatted at 2026-08-09 16:29:33 on dist-test-slave-j2vl"
server_key: "c17b34cec677d5f434c3ad6aec63285f"
server_key_iv: "cdcce27ee021983be7fa2fd1a43538e5"
server_key_version: "encryptionkey@0"
I20260809 16:29:33.481916 1562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:29:33.482877 1588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:33.483106 1562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:33.483220 1562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "e70fb780c0f243c1bcb8099b4492a7d5"
format_stamp: "Formatted at 2026-08-09 16:29:33 on dist-test-slave-j2vl"
server_key: "c17b34cec677d5f434c3ad6aec63285f"
server_key_iv: "cdcce27ee021983be7fa2fd1a43538e5"
server_key_version: "encryptionkey@0"
I20260809 16:29:33.483299 1562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:33.498090 1562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:33.500527 1562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:33.500699 1562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:33.501245 1562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:33.501310 1562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:33.501374 1562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:33.501405 1562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:29:33 dist-test-slave-j2vl krb5kdc[1140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292973, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:33.509014 1562 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:46393
I20260809 16:29:33.509075 1786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:46393 every 8 connection(s)
I20260809 16:29:33.509613 1562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:29:33.512225 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.501863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:35847 (local address 127.30.23.190:34131)
0809 16:29:33.501950 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.501952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.502144 (+ 192us) server_negotiation.cc:407] Connection header received
0809 16:29:33.502645 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.502647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.502662 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.502740 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:33.503496 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.504345 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.506093 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.506450 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.506523 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.508956 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:33.508958 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:33.508958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:33.508966 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:33.510767 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.511201 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.511203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.511203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.511269 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.511539 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.511540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.511541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.511672 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:33.511750 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.512045 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.512133 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 16:29:33.512401 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.501790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34131 (local address 127.30.23.130:35847)
0809 16:29:33.502007 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.502048 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.502541 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.502842 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.502847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.503181 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:33.503377 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.503382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.504456 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.504458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.505995 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.505999 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.506561 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.506562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.506736 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.507397 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:33.507405 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:33.508854 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:33.510876 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.510883 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.510886 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.511120 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.511346 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.511347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.511347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.511462 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.511740 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:33.511748 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:33.511897 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.512007 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.512169 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":124,"threads_started":1}
I20260809 16:29:33.512917 1787 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34131
I20260809 16:29:33.513027 1787 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:33.513212 1787 heartbeater.cc:511] Master 127.30.23.190:34131 requested a full tablet report, sending...
I20260809 16:29:33.513659 1218 ts_manager.cc:194] Registered new tserver with Master: e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130:46393)
I20260809 16:29:33.514189 1218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:35847
I20260809 16:29:33.516971 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 1562
I20260809 16:29:33.517087 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:29:33.517289 30814 external_mini_cluster.cc:1428] Setting key eb511ee4ec5dffde1ee98740c6490275
I20260809 16:29:33.520342 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.514102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:49005 (local address 127.30.23.190:34131)
0809 16:29:33.514141 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.514143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.514245 (+ 102us) server_negotiation.cc:407] Connection header received
0809 16:29:33.514316 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.514317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.514327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.514382 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:33.514743 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.515219 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.515953 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.516156 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.516230 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.517387 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:33.517389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:33.517389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:33.517398 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:33.518853 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.519319 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.519320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.519320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.519380 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.519633 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.519634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.519634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.519816 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:33.519883 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.520203 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.520279 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:29:33.520406 1798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.514043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34131 (local address 127.30.23.130:49005)
0809 16:29:33.514181 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.514192 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.514263 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.514418 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.514420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.514601 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:33.514652 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.514654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.515318 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.515319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.515850 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.515851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.516228 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.516229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.516307 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.516794 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:33.516802 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:33.517233 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:33.518955 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.518956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.518956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.519204 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.519458 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.519459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.519460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.519536 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.519973 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:33.519975 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:33.520035 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.520124 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.520250 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:29:33.550278 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34131
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:33.637756 1805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:33.637976 1805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:33.638002 1805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:33.639122 1805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:33.639158 1805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:33.639179 1805 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 16:29:33.639191 1805 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:29:33.639209 1805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:29:33.640616 1805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.23.190:34131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:33.640810 1805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:33.640995 1805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:33.646657 1811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:33.646765 1817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:33.646765 1814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:33.646945 1805 server_base.cc:1034] running on GCE node
I20260809 16:29:33.647264 1805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:33.647564 1805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:33.648763 1805 hybrid_clock.cc:648] HybridClock initialized: now 1786292973648746 us; error 35 us; skew 500 ppm
Aug 09 16:29:33 dist-test-slave-j2vl krb5kdc[1140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292973, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:33.651652 1805 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:29:33.652177 1805 webserver.cc:460] Webserver started at http://127.30.23.131:40813/ using document root <none> and password file <none>
I20260809 16:29:33.652426 1805 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:33.652500 1805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:33.652680 1805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:33.653861 1805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "31a2e03b656c4fc5ba79035b60086f2c"
format_stamp: "Formatted at 2026-08-09 16:29:33 on dist-test-slave-j2vl"
server_key: "4bcf68d87905712f6dc00f89eef2c98d"
server_key_iv: "e4fc71801b94aeaebaa0a3942cb00e8d"
server_key_version: "encryptionkey@0"
I20260809 16:29:33.654250 1805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "31a2e03b656c4fc5ba79035b60086f2c"
format_stamp: "Formatted at 2026-08-09 16:29:33 on dist-test-slave-j2vl"
server_key: "4bcf68d87905712f6dc00f89eef2c98d"
server_key_iv: "e4fc71801b94aeaebaa0a3942cb00e8d"
server_key_version: "encryptionkey@0"
I20260809 16:29:33.655728 1805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:29:33.656682 1832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:33.656947 1805 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:33.657075 1805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "31a2e03b656c4fc5ba79035b60086f2c"
format_stamp: "Formatted at 2026-08-09 16:29:33 on dist-test-slave-j2vl"
server_key: "4bcf68d87905712f6dc00f89eef2c98d"
server_key_iv: "e4fc71801b94aeaebaa0a3942cb00e8d"
server_key_version: "encryptionkey@0"
I20260809 16:29:33.657160 1805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:33.675290 1805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:33.677891 1805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:33.678084 1805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:33.678758 1805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:33.678813 1805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:33.678848 1805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:33.678866 1805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:29:33 dist-test-slave-j2vl krb5kdc[1140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292973, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:33.686515 1805 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:41839
I20260809 16:29:33.686650 2029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:41839 every 8 connection(s)
I20260809 16:29:33.686995 1805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:29:33.688982 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.679215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:48621 (local address 127.30.23.190:34131)
0809 16:29:33.679288 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.679290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.679562 (+ 272us) server_negotiation.cc:407] Connection header received
0809 16:29:33.680086 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.680087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.680102 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.680181 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:33.680891 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.681625 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.682668 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.682879 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.682956 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.685741 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:33.685743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:33.685743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:33.685751 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:33.687567 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.687981 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.687982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.687983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.688046 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.688315 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.688316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.688316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.688441 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:33.688519 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.688799 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.688889 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:29:33.689182 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.679206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34131 (local address 127.30.23.131:48621)
0809 16:29:33.679422 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.679463 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.679983 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.680271 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.680274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.680572 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:33.680787 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.680791 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.681745 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.681747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.682561 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.682563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.682964 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.682965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.683132 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.683706 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:33.683715 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:33.685623 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:33.687670 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.687675 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.687678 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.687901 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.688121 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.688121 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.688122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.688241 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.688507 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:33.688514 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:33.688658 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.688775 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.688942 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":144,"threads_started":1}
I20260809 16:29:33.689692 2030 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34131
I20260809 16:29:33.689793 2030 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:33.689975 2030 heartbeater.cc:511] Master 127.30.23.190:34131 requested a full tablet report, sending...
I20260809 16:29:33.690389 1219 ts_manager.cc:194] Registered new tserver with Master: 31a2e03b656c4fc5ba79035b60086f2c (127.30.23.131:41839)
I20260809 16:29:33.690881 1219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:48621
I20260809 16:29:33.696189 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.690911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:59211 (local address 127.30.23.190:34131)
0809 16:29:33.690961 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.690963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.691066 (+ 103us) server_negotiation.cc:407] Connection header received
0809 16:29:33.691140 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.691141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.691151 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.691197 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:33.691526 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.692058 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.692732 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.692918 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.692971 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.693892 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:33.693894 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:33.693894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:33.693900 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:33.695304 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.695606 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.695607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.695607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.695656 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.695813 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.695814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.695814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.695913 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:33.695963 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.696071 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.696128 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:29:33.696197 2039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.690796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34131 (local address 127.30.23.131:59211)
0809 16:29:33.691002 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.691015 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.691081 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.691211 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.691212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.691395 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:33.691451 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.691453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.692144 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.692144 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.692662 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.692663 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.692983 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.692984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.693038 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.693542 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:33.693546 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:33.693816 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:33.695388 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.695388 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.695389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.695548 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.695712 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.695713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.695713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.695762 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.695968 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:33.695969 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:33.696005 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.696056 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.696133 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":133,"threads_started":1}
I20260809 16:29:33.697137 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 1805
I20260809 16:29:33.697252 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:29:33.697486 30814 external_mini_cluster.cc:1428] Setting key 61e542f2532f5b0547ea25a3c4d8e3a7
I20260809 16:29:33.698383 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:29:33 dist-test-slave-j2vl krb5kdc[1140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:29:33 dist-test-slave-j2vl krb5kdc[1140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:33.724627 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.716201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59294 (local address 127.30.23.190:34131)
0809 16:29:33.716287 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.716289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.716296 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:29:33.716322 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.716323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.716332 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.716395 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:33.717029 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.717601 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.718440 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.718636 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.718718 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.721721 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:33.721723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:33.721724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:33.721740 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:33.723278 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.723716 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.723718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.723718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.723783 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.724015 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.724017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.724017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.724235 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:33.724332 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.724458 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.724529 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 16:29:33.726807 1219 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:29:33.728449 1219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:29:33.735737 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.731310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45619 (local address 127.0.0.1:40970)
0809 16:29:33.731741 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.731755 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.731855 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.732251 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.732252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.732254 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:33.732307 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.732309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.733210 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.733211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.734367 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.734370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.735380 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.735381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.735479 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.735495 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.735577 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.735642 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":86,"threads_started":1}
I20260809 16:29:33.735771 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.731417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40970 (local address 127.30.23.129:45619)
0809 16:29:33.731643 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.731648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.731963 (+ 315us) server_negotiation.cc:407] Connection header received
0809 16:29:33.732016 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.732018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.732036 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.732160 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:33.732391 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.733109 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.734461 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.735281 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.735366 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.735440 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.735605 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.735656 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":103,"threads_started":1}
I20260809 16:29:33.736172 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.731502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38882 (local address 127.30.23.131:41839)
0809 16:29:33.731721 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.731725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.731736 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:29:33.731779 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.731781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.731799 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.731916 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:33.732211 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.732963 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.734616 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.735384 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.735445 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.735492 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.735783 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.735834 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":79,"threads_started":1}
I20260809 16:29:33.736371 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.731305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:41839 (local address 127.0.0.1:38882)
0809 16:29:33.731484 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.731527 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.731597 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.732018 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.732022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.732025 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:33.732121 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.732124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.733074 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.733075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.734522 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.734526 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.735534 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.735535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.735618 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.735623 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.735675 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.735722 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":70,"threads_started":1}
I20260809 16:29:33.736806 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.731512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:46393 (local address 127.0.0.1:58794)
0809 16:29:33.733129 (+ 1617us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.733140 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.733190 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.733473 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.733474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.733476 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:33.733517 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.733518 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.734759 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.734760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.735556 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.735558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.736580 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.736581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.736643 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.736647 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.736697 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.736740 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1592,"thread_start_us":83,"threads_started":1}
I20260809 16:29:33.736680 1442 tablet_service.cc:1467] Processing CreateTablet for tablet eed54efc0a4b4556874aabcce8d40485 (DEFAULT_TABLE table=test-table [id=36a4b880fe844934904ad1ad8d212ae4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:33.737078 2065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.731761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58794 (local address 127.30.23.130:46393)
0809 16:29:33.731998 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.732002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.733253 (+ 1251us) server_negotiation.cc:407] Connection header received
0809 16:29:33.733304 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.733307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.733322 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.733424 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:33.733606 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.734662 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.735649 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.736422 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.736492 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.736543 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.736918 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.736964 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":121,"threads_started":1}
I20260809 16:29:33.737124 1442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eed54efc0a4b4556874aabcce8d40485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:33.737058 1913 tablet_service.cc:1467] Processing CreateTablet for tablet eed54efc0a4b4556874aabcce8d40485 (DEFAULT_TABLE table=test-table [id=36a4b880fe844934904ad1ad8d212ae4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:33.737511 1913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eed54efc0a4b4556874aabcce8d40485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:33.740336 2067 tablet_bootstrap.cc:492] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c: Bootstrap starting.
I20260809 16:29:33.740516 2068 tablet_bootstrap.cc:492] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1: Bootstrap starting.
I20260809 16:29:33.741240 2067 tablet_bootstrap.cc:654] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:33.741241 2068 tablet_bootstrap.cc:654] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:33.741289 1675 tablet_service.cc:1467] Processing CreateTablet for tablet eed54efc0a4b4556874aabcce8d40485 (DEFAULT_TABLE table=test-table [id=36a4b880fe844934904ad1ad8d212ae4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:33.741602 2067 log.cc:824] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:33.741767 2068 log.cc:824] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:33.741843 1675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eed54efc0a4b4556874aabcce8d40485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:33.742483 2068 tablet_bootstrap.cc:492] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1: No bootstrap required, opened a new log
I20260809 16:29:33.742497 2067 tablet_bootstrap.cc:492] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c: No bootstrap required, opened a new log
I20260809 16:29:33.742569 2067 ts_tablet_manager.cc:1397] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:29:33.742573 2068 ts_tablet_manager.cc:1397] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:29:33.744097 2068 raft_consensus.cc:348] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } }
I20260809 16:29:33.744261 2068 raft_consensus.cc:374] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:33.744314 2068 raft_consensus.cc:729] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a6b20d23404b649a1a94d7d47a01c1, State: Initialized, Role: FOLLOWER
I20260809 16:29:33.744455 2068 consensus_queue.cc:260] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } }
I20260809 16:29:33.744812 2068 ts_tablet_manager.cc:1428] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:29:33.744669 2067 raft_consensus.cc:348] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } }
I20260809 16:29:33.744837 1551 heartbeater.cc:503] Master 127.30.23.190:34131 was elected leader, sending a full tablet report...
I20260809 16:29:33.744874 2067 raft_consensus.cc:374] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:33.744910 2067 raft_consensus.cc:729] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a2e03b656c4fc5ba79035b60086f2c, State: Initialized, Role: FOLLOWER
I20260809 16:29:33.745050 2067 consensus_queue.cc:260] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } }
I20260809 16:29:33.745381 2067 ts_tablet_manager.cc:1428] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:29:33.745395 2030 heartbeater.cc:503] Master 127.30.23.190:34131 was elected leader, sending a full tablet report...
I20260809 16:29:33.746055 2076 tablet_bootstrap.cc:492] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5: Bootstrap starting.
I20260809 16:29:33.746728 2076 tablet_bootstrap.cc:654] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:33.747084 2076 log.cc:824] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:33.747951 2076 tablet_bootstrap.cc:492] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5: No bootstrap required, opened a new log
I20260809 16:29:33.748013 2076 ts_tablet_manager.cc:1397] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:29:33.749619 2076 raft_consensus.cc:348] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } }
I20260809 16:29:33.749751 2076 raft_consensus.cc:374] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:33.749783 2076 raft_consensus.cc:729] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70fb780c0f243c1bcb8099b4492a7d5, State: Initialized, Role: FOLLOWER
I20260809 16:29:33.749922 2076 consensus_queue.cc:260] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } }
I20260809 16:29:33.750216 2076 ts_tablet_manager.cc:1428] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 16:29:33.750303 1787 heartbeater.cc:503] Master 127.30.23.190:34131 was elected leader, sending a full tablet report...
W20260809 16:29:33.760568 1788 tablet.cc:2369] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:29:33.833496 2074 raft_consensus.cc:482] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:29:33.833616 2074 raft_consensus.cc:504] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } }
I20260809 16:29:33.834022 2074 leader_election.cc:288] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31a2e03b656c4fc5ba79035b60086f2c (127.30.23.131:41839), e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130:46393)
I20260809 16:29:33.837695 1384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.834232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:41839 (local address 127.30.23.129:42221)
0809 16:29:33.834287 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.834301 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.834352 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.834598 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.834600 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.834602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:33.834680 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.834682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.835450 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.835451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.836484 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.836486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.837446 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.837447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.837534 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.837537 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.837583 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.837618 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 16:29:33.837741 2065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.834532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:56027 (local address 127.30.23.130:46393)
0809 16:29:33.834592 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.834594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.834604 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:29:33.834629 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.834630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.834639 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.834721 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:33.834925 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.835760 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.836882 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.837272 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.837382 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.837394 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.837554 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.837618 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:29:33.837702 2080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.834375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:46393 (local address 127.30.23.129:56027)
0809 16:29:33.834513 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.834522 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.834570 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.834727 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.834728 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.834729 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:33.834832 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.834834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.835862 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.835863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.836795 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.836797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.837379 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.837380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.837455 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.837460 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.837512 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.837560 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260809 16:29:33.837878 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.834339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:42221 (local address 127.30.23.131:41839)
0809 16:29:33.834392 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.834395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.834405 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:29:33.834440 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.834442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.834454 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.834534 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:33.834786 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.835338 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.836595 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.837316 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.837426 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.837437 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.837743 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.837793 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:29:33.838205 1946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed54efc0a4b4556874aabcce8d40485" candidate_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a2e03b656c4fc5ba79035b60086f2c" is_pre_election: true
I20260809 16:29:33.838348 1710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed54efc0a4b4556874aabcce8d40485" candidate_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" is_pre_election: true
I20260809 16:29:33.838400 1946 raft_consensus.cc:2454] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2a6b20d23404b649a1a94d7d47a01c1 in term 0.
I20260809 16:29:33.838485 1710 raft_consensus.cc:2454] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2a6b20d23404b649a1a94d7d47a01c1 in term 0.
I20260809 16:29:33.838722 1373 leader_election.cc:302] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a2e03b656c4fc5ba79035b60086f2c, a2a6b20d23404b649a1a94d7d47a01c1; no voters:
I20260809 16:29:33.838853 2074 raft_consensus.cc:2781] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:29:33.838915 2074 raft_consensus.cc:482] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:29:33.838945 2074 raft_consensus.cc:3037] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:33.839912 2074 raft_consensus.cc:504] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } }
I20260809 16:29:33.840065 2074 leader_election.cc:288] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [CANDIDATE]: Term 1 election: Requested vote from peers 31a2e03b656c4fc5ba79035b60086f2c (127.30.23.131:41839), e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130:46393)
I20260809 16:29:33.840271 1710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed54efc0a4b4556874aabcce8d40485" candidate_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70fb780c0f243c1bcb8099b4492a7d5"
I20260809 16:29:33.840269 1946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eed54efc0a4b4556874aabcce8d40485" candidate_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a2e03b656c4fc5ba79035b60086f2c"
I20260809 16:29:33.840353 1946 raft_consensus.cc:3037] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:33.840353 1710 raft_consensus.cc:3037] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:33.841259 1946 raft_consensus.cc:2454] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2a6b20d23404b649a1a94d7d47a01c1 in term 1.
I20260809 16:29:33.841259 1710 raft_consensus.cc:2454] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2a6b20d23404b649a1a94d7d47a01c1 in term 1.
I20260809 16:29:33.841454 1373 leader_election.cc:302] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a2e03b656c4fc5ba79035b60086f2c, a2a6b20d23404b649a1a94d7d47a01c1; no voters:
I20260809 16:29:33.841549 2074 raft_consensus.cc:2781] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:33.841749 2074 raft_consensus.cc:686] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 LEADER]: Becoming Leader. State: Replica: a2a6b20d23404b649a1a94d7d47a01c1, State: Running, Role: LEADER
I20260809 16:29:33.841856 2074 consensus_queue.cc:237] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } }
I20260809 16:29:33.842854 1219 catalog_manager.cc:5495] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129). New cstate: current_term: 1 leader_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } health_report { overall_health: HEALTHY } } }
W20260809 16:29:33.844254 1552 tablet.cc:2369] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 16:29:33 dist-test-slave-j2vl krb5kdc[1140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292973, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:33.878443 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.870265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59298 (local address 127.30.23.190:34131)
0809 16:29:33.870336 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.870338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.870534 (+ 196us) server_negotiation.cc:407] Connection header received
0809 16:29:33.870621 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.870621 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.870632 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.870690 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:33.871181 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.871688 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.872384 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.872770 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.872913 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.875465 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:33.875466 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:33.875467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:33.875474 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:33.876994 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.877612 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.877613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.877614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.877673 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:33.877924 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:33.877925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:33.877925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:33.878041 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:33.878098 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.878303 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.878374 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:29:33.878654 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.870221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34131 (local address 127.0.0.1:59298)
0809 16:29:33.870453 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.870463 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.870557 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.870695 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.870696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.871041 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:33.871090 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.871093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.871773 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.871774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.872296 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.872298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.872915 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.872916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.872985 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.873614 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:33.873620 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:33.875302 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:33.877095 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.877107 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.877110 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.877508 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.877760 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:33.877761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:33.877762 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:33.877848 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:33.878114 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:33.878122 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:33.878193 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.878290 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.878478 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":156,"threads_started":1}
I20260809 16:29:33.881079 1219 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:59298:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 16:29:33.881415 1219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:29:33.882961 1442 tablet_service.cc:1467] Processing CreateTablet for tablet 19df79d740e746be9c258d1f58cbfbc3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=050d8d4927054b21b175ff8e06e72358]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:29:33.883024 1675 tablet_service.cc:1467] Processing CreateTablet for tablet 19df79d740e746be9c258d1f58cbfbc3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=050d8d4927054b21b175ff8e06e72358]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:29:33.883180 1442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19df79d740e746be9c258d1f58cbfbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:33.883204 1675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19df79d740e746be9c258d1f58cbfbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:33.884377 2076 tablet_bootstrap.cc:492] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5: Bootstrap starting.
I20260809 16:29:33.884491 1913 tablet_service.cc:1467] Processing CreateTablet for tablet 19df79d740e746be9c258d1f58cbfbc3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=050d8d4927054b21b175ff8e06e72358]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:29:33.884649 1913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19df79d740e746be9c258d1f58cbfbc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:33.885121 2076 tablet_bootstrap.cc:654] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:33.885458 2068 tablet_bootstrap.cc:492] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1: Bootstrap starting.
I20260809 16:29:33.885972 2068 tablet_bootstrap.cc:654] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:33.886204 2067 tablet_bootstrap.cc:492] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c: Bootstrap starting.
I20260809 16:29:33.886794 2067 tablet_bootstrap.cc:654] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:33.886811 2076 tablet_bootstrap.cc:492] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5: No bootstrap required, opened a new log
I20260809 16:29:33.886852 2076 ts_tablet_manager.cc:1397] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:29:33.887023 2076 raft_consensus.cc:348] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.887132 2076 raft_consensus.cc:374] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:33.887162 2076 raft_consensus.cc:729] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70fb780c0f243c1bcb8099b4492a7d5, State: Initialized, Role: FOLLOWER
I20260809 16:29:33.887212 2076 consensus_queue.cc:260] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.887326 2076 ts_tablet_manager.cc:1428] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:29:33.887471 2079 tablet_replica.cc:439] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } }
I20260809 16:29:33.887563 2079 tablet_replica.cc:442] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:33.887809 2067 tablet_bootstrap.cc:492] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c: No bootstrap required, opened a new log
I20260809 16:29:33.887846 2067 ts_tablet_manager.cc:1397] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:29:33.888007 2067 raft_consensus.cc:348] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.888078 2067 raft_consensus.cc:374] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:33.888093 2067 raft_consensus.cc:729] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a2e03b656c4fc5ba79035b60086f2c, State: Initialized, Role: FOLLOWER
I20260809 16:29:33.888146 2067 consensus_queue.cc:260] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.888247 2067 ts_tablet_manager.cc:1428] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:29:33.888396 2075 tablet_replica.cc:439] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } }
I20260809 16:29:33.888484 2075 tablet_replica.cc:442] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:33.888980 2068 tablet_bootstrap.cc:492] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1: No bootstrap required, opened a new log
I20260809 16:29:33.889056 2068 ts_tablet_manager.cc:1397] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 16:29:33.889248 2068 raft_consensus.cc:348] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.889376 2068 raft_consensus.cc:374] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:33.889410 2068 raft_consensus.cc:729] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a6b20d23404b649a1a94d7d47a01c1, State: Initialized, Role: FOLLOWER
I20260809 16:29:33.889467 2068 consensus_queue.cc:260] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.889599 2068 ts_tablet_manager.cc:1428] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:33.889679 2074 tablet_replica.cc:439] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } }
I20260809 16:29:33.889783 2074 tablet_replica.cc:442] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:33.922407 2079 raft_consensus.cc:482] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:29:33.922540 2079 raft_consensus.cc:504] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.922992 2079 leader_election.cc:288] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129:45619), 31a2e03b656c4fc5ba79035b60086f2c (127.30.23.131:41839)
I20260809 16:29:33.926177 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.923257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:56843 (local address 127.30.23.129:45619)
0809 16:29:33.923299 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.923301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.923312 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:29:33.923355 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.923356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.923366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.923442 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:33.923635 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.924166 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.925236 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.925773 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.925866 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.925877 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.926067 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.926110 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:29:33.926177 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.923160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45619 (local address 127.30.23.130:56843)
0809 16:29:33.923215 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.923227 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.923268 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.923474 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.923475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.923477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:33.923542 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.923543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.924258 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.924260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.925133 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.925136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.925867 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.925868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.925969 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.925973 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.926057 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.926100 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:29:33.926244 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.923297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:33373 (local address 127.30.23.131:41839)
0809 16:29:33.923356 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.923358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.923522 (+ 164us) server_negotiation.cc:407] Connection header received
0809 16:29:33.923582 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.923583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.923593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.923641 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:33.923801 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.924360 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.925430 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.925866 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.925946 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.925955 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.926134 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.926170 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:29:33.926384 2097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.923229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:41839 (local address 127.30.23.130:33373)
0809 16:29:33.923463 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.923473 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.923528 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.923642 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.923642 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.923643 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:33.923717 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.923718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.924441 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.924442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.925225 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.925227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.925979 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.925980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.926059 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.926062 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.926128 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.926165 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":162,"threads_started":1}
I20260809 16:29:33.926659 1946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19df79d740e746be9c258d1f58cbfbc3" candidate_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a2e03b656c4fc5ba79035b60086f2c" is_pre_election: true
I20260809 16:29:33.926793 1946 raft_consensus.cc:2454] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e70fb780c0f243c1bcb8099b4492a7d5 in term 0.
I20260809 16:29:33.926764 1472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19df79d740e746be9c258d1f58cbfbc3" candidate_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" is_pre_election: true
I20260809 16:29:33.926921 1472 raft_consensus.cc:2454] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e70fb780c0f243c1bcb8099b4492a7d5 in term 0.
I20260809 16:29:33.926975 1593 leader_election.cc:302] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a2e03b656c4fc5ba79035b60086f2c, e70fb780c0f243c1bcb8099b4492a7d5; no voters:
I20260809 16:29:33.927120 2079 raft_consensus.cc:2781] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:29:33.927168 2079 raft_consensus.cc:482] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:29:33.927183 2079 raft_consensus.cc:3037] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:33.927889 2079 raft_consensus.cc:504] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.928035 2079 leader_election.cc:288] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [CANDIDATE]: Term 1 election: Requested vote from peers a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129:45619), 31a2e03b656c4fc5ba79035b60086f2c (127.30.23.131:41839)
I20260809 16:29:33.928179 1472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19df79d740e746be9c258d1f58cbfbc3" candidate_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a6b20d23404b649a1a94d7d47a01c1"
I20260809 16:29:33.928252 1472 raft_consensus.cc:3037] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:33.928241 1946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19df79d740e746be9c258d1f58cbfbc3" candidate_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31a2e03b656c4fc5ba79035b60086f2c"
I20260809 16:29:33.928323 1946 raft_consensus.cc:3037] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:33.928983 1946 raft_consensus.cc:2454] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e70fb780c0f243c1bcb8099b4492a7d5 in term 1.
I20260809 16:29:33.929000 1472 raft_consensus.cc:2454] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e70fb780c0f243c1bcb8099b4492a7d5 in term 1.
I20260809 16:29:33.929210 1595 leader_election.cc:302] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2a6b20d23404b649a1a94d7d47a01c1, e70fb780c0f243c1bcb8099b4492a7d5; no voters:
I20260809 16:29:33.929363 2079 raft_consensus.cc:2781] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:33.929574 2079 raft_consensus.cc:686] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 LEADER]: Becoming Leader. State: Replica: e70fb780c0f243c1bcb8099b4492a7d5, State: Running, Role: LEADER
I20260809 16:29:33.929675 2079 consensus_queue.cc:237] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.930105 2099 tablet_replica.cc:439] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5: TxnStatusTablet state changed. Reason: New leader e70fb780c0f243c1bcb8099b4492a7d5. Latest consensus state: current_term: 1 leader_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } }
I20260809 16:29:33.930225 2099 tablet_replica.cc:442] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:29:33.930464 2101 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 16:29:33.930495 1218 catalog_manager.cc:5495] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130). New cstate: current_term: 1 leader_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } health_report { overall_health: UNKNOWN } } }
I20260809 16:29:33.930586 2101 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 16:29:33.936329 2065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.933646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58796 (local address 127.30.23.130:46393)
0809 16:29:33.933685 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.933687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.933738 (+ 51us) server_negotiation.cc:407] Connection header received
0809 16:29:33.933840 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.933840 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.933854 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.933915 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:33.934038 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.934487 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.935563 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.935741 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.935802 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.936024 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:33.936109 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:33.936165 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.936264 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.936282 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:29:33.936400 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.933601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:46393 (local address 127.0.0.1:58796)
0809 16:29:33.933681 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.933695 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.933762 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.933930 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.933931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.933932 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:29:33.933981 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.933984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.934564 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.934565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.935476 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.935479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.935825 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.935826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.935905 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.935923 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:29:33.936179 (+ 256us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:29:33.936189 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.936263 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.936322 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 16:29:33.937685 2033 tablet.cc:2369] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:29:33.938220 1946 raft_consensus.cc:1261] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Refusing update from remote peer e70fb780c0f243c1bcb8099b4492a7d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:33.938252 1472 raft_consensus.cc:1261] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Refusing update from remote peer e70fb780c0f243c1bcb8099b4492a7d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:33.938530 2099 consensus_queue.cc:1035] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:33.938685 2079 consensus_queue.cc:1035] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:33.939889 2075 tablet_replica.cc:439] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c: TxnStatusTablet state changed. Reason: New leader e70fb780c0f243c1bcb8099b4492a7d5. Latest consensus state: current_term: 1 leader_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } }
I20260809 16:29:33.939909 2074 tablet_replica.cc:439] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1: TxnStatusTablet state changed. Reason: New leader e70fb780c0f243c1bcb8099b4492a7d5. Latest consensus state: current_term: 1 leader_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } }
I20260809 16:29:33.939981 2075 tablet_replica.cc:442] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:33.939986 2074 tablet_replica.cc:442] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:33.941092 2105 mvcc.cc:205] Tried to move back new op lower bound from 7316656021248368640 to 7316656021216444416. Current Snapshot: MvccSnapshot[applied={T|T < 7316656021248368640}]
I20260809 16:29:33.941159 2079 tablet_replica.cc:439] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } }
I20260809 16:29:33.941241 2079 tablet_replica.cc:442] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:29:33.941579 2075 tablet_replica.cc:439] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } }
I20260809 16:29:33.941655 2075 tablet_replica.cc:442] T 19df79d740e746be9c258d1f58cbfbc3 P 31a2e03b656c4fc5ba79035b60086f2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:33.941255 2099 tablet_replica.cc:439] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } }
I20260809 16:29:33.941823 2074 tablet_replica.cc:439] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } } }
I20260809 16:29:33.941886 2074 tablet_replica.cc:442] T 19df79d740e746be9c258d1f58cbfbc3 P a2a6b20d23404b649a1a94d7d47a01c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:29:33.941885 2099 tablet_replica.cc:442] T 19df79d740e746be9c258d1f58cbfbc3 P e70fb780c0f243c1bcb8099b4492a7d5: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:29:33.942318 2109 mvcc.cc:205] Tried to move back new op lower bound from 7316656021248368640 to 7316656021216444416. Current Snapshot: MvccSnapshot[applied={T|T < 7316656021248368640}]
I20260809 16:29:33.942453 2106 mvcc.cc:205] Tried to move back new op lower bound from 7316656021248368640 to 7316656021216444416. Current Snapshot: MvccSnapshot[applied={T|T < 7316656021248368640}]
I20260809 16:29:33.946391 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.943898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.30.23.129:45619)
0809 16:29:33.943947 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.943949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.943957 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:29:33.944014 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.944014 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.944029 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.944114 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:33.944267 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.944725 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.945557 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.945823 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.945903 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.946072 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:33.946181 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:33.946273 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.946296 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.946319 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:29:33.951571 2065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.948875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:51827 (local address 127.30.23.130:46393)
0809 16:29:33.948918 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.948920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.949037 (+ 117us) server_negotiation.cc:407] Connection header received
0809 16:29:33.949108 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.949109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.949131 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.949198 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:33.949312 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.950026 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.950765 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.951131 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.951204 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.951364 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:33.951395 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:33.951453 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.951486 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.951506 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:29:33.951581 2125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.948802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:46393 (local address 127.30.23.129:51827)
0809 16:29:33.948961 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.948971 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.949050 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.949199 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.949200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.949201 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:29:33.949252 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.949253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.950108 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.950109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.950679 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.950680 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.951204 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.951205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.951259 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.951274 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:29:33.951453 (+ 179us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:29:33.951453 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.951483 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.951514 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260809 16:29:33.953625 1946 raft_consensus.cc:1261] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Refusing update from remote peer a2a6b20d23404b649a1a94d7d47a01c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:33.953697 1710 raft_consensus.cc:1261] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Refusing update from remote peer a2a6b20d23404b649a1a94d7d47a01c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:33.953894 2074 consensus_queue.cc:1035] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:33.954034 2081 consensus_queue.cc:1035] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:33.963244 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.960296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:46937 (local address 127.30.23.129:45619)
0809 16:29:33.960340 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:33.960342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:33.960349 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:29:33.960370 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:33.960371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:33.960382 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:33.960453 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:33.960595 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.961142 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.962490 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.962707 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.962795 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.962991 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:33.963026 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:33.963078 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:33.963141 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 16:29:33.963163 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:29:33.963250 1798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:33.960046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45619 (local address 127.30.23.130:46937)
0809 16:29:33.960109 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:33.960122 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:33.960164 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:33.960473 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:33.960474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:33.960476 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:29:33.960542 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.960544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.961237 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.961238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.962375 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:33.962378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:33.962796 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:33.962797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:33.962867 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:33.962890 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:29:33.963084 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:29:33.963086 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:29:33.963138 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:29:33.963183 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 16:29:33.972101 1218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:33.972239 1218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:33.973029 1218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb: Sending DeleteTablet for 3 replicas of tablet eed54efc0a4b4556874aabcce8d40485
I20260809 16:29:33.973372 1442 tablet_service.cc:1514] Processing DeleteTablet for tablet eed54efc0a4b4556874aabcce8d40485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:33 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:40970
I20260809 16:29:33.973496 1913 tablet_service.cc:1514] Processing DeleteTablet for tablet eed54efc0a4b4556874aabcce8d40485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:33 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:38882
I20260809 16:29:33.973733 2138 tablet_replica.cc:330] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c: stopping tablet replica
I20260809 16:29:33.973721 1675 tablet_service.cc:1514] Processing DeleteTablet for tablet eed54efc0a4b4556874aabcce8d40485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:33 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:58794
I20260809 16:29:33.973924 2138 raft_consensus.cc:2229] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:33.974025 2138 raft_consensus.cc:2258] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:33.974236 1218 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:29:33.974388 1218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:29:33.974556 2138 ts_tablet_manager.cc:1905] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:33.976052 1913 tablet_service.cc:1467] Processing CreateTablet for tablet 57d127e7f7cc4755ae2af803e9a829a1 (DEFAULT_TABLE table=test-table [id=74afafd21683481fb2f7e83219050271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:33.976183 1913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57d127e7f7cc4755ae2af803e9a829a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:33.976569 1442 tablet_service.cc:1467] Processing CreateTablet for tablet 57d127e7f7cc4755ae2af803e9a829a1 (DEFAULT_TABLE table=test-table [id=74afafd21683481fb2f7e83219050271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:33.976696 1442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57d127e7f7cc4755ae2af803e9a829a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:33.977481 2067 tablet_bootstrap.cc:492] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c: Bootstrap starting.
I20260809 16:29:33.977865 2067 tablet_bootstrap.cc:654] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:33.978170 2068 tablet_bootstrap.cc:492] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1: Bootstrap starting.
I20260809 16:29:33.978585 2068 tablet_bootstrap.cc:654] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:33.978578 1675 tablet_service.cc:1467] Processing CreateTablet for tablet 57d127e7f7cc4755ae2af803e9a829a1 (DEFAULT_TABLE table=test-table [id=74afafd21683481fb2f7e83219050271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:33.978703 1675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57d127e7f7cc4755ae2af803e9a829a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:33.979075 2138 ts_tablet_manager.cc:1918] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:29:33.979146 2138 log.cc:1196] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/eed54efc0a4b4556874aabcce8d40485
I20260809 16:29:33.979426 2138 ts_tablet_manager.cc:1939] T eed54efc0a4b4556874aabcce8d40485 P 31a2e03b656c4fc5ba79035b60086f2c: Deleting consensus metadata
I20260809 16:29:33.979825 1194 catalog_manager.cc:4841] TS 31a2e03b656c4fc5ba79035b60086f2c (127.30.23.131:41839): tablet eed54efc0a4b4556874aabcce8d40485 (table test-table [id=36a4b880fe844934904ad1ad8d212ae4]) successfully deleted
I20260809 16:29:33.980055 2076 tablet_bootstrap.cc:492] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5: Bootstrap starting.
I20260809 16:29:33.980444 2067 tablet_bootstrap.cc:492] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c: No bootstrap required, opened a new log
I20260809 16:29:33.980480 2076 tablet_bootstrap.cc:654] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:33.980485 2067 ts_tablet_manager.cc:1397] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:29:33.980628 2067 raft_consensus.cc:348] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.980691 2067 raft_consensus.cc:374] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:33.980711 2067 raft_consensus.cc:729] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a2e03b656c4fc5ba79035b60086f2c, State: Initialized, Role: FOLLOWER
I20260809 16:29:33.980754 2067 consensus_queue.cc:260] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.980840 2067 ts_tablet_manager.cc:1428] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:29:33.981585 2068 tablet_bootstrap.cc:492] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1: No bootstrap required, opened a new log
I20260809 16:29:33.981637 2068 ts_tablet_manager.cc:1397] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 16:29:33.981770 2068 raft_consensus.cc:348] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.981841 2068 raft_consensus.cc:374] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:33.981863 2068 raft_consensus.cc:729] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a6b20d23404b649a1a94d7d47a01c1, State: Initialized, Role: FOLLOWER
I20260809 16:29:33.981912 2068 consensus_queue.cc:260] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.981930 2076 tablet_bootstrap.cc:492] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5: No bootstrap required, opened a new log
I20260809 16:29:33.981976 2076 ts_tablet_manager.cc:1397] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260809 16:29:33.981999 2068 ts_tablet_manager.cc:1428] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:33.982118 2076 raft_consensus.cc:348] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.982201 2076 raft_consensus.cc:374] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:33.982223 2076 raft_consensus.cc:729] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70fb780c0f243c1bcb8099b4492a7d5, State: Initialized, Role: FOLLOWER
I20260809 16:29:33.982275 2076 consensus_queue.cc:260] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:33.982379 2076 ts_tablet_manager.cc:1428] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:33.983306 2137 tablet_replica.cc:330] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1: stopping tablet replica
I20260809 16:29:33.983435 2137 raft_consensus.cc:2229] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:29:33.983548 2137 raft_consensus.cc:2258] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:33.983811 2142 tablet_replica.cc:330] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5: stopping tablet replica
I20260809 16:29:33.984107 2137 ts_tablet_manager.cc:1905] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:33.985051 2142 raft_consensus.cc:2229] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:33.985188 2142 raft_consensus.cc:2258] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:33.986449 2142 ts_tablet_manager.cc:1905] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:33.987448 2137 ts_tablet_manager.cc:1918] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:29:33.988303 2137 log.cc:1196] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/wal/wals/eed54efc0a4b4556874aabcce8d40485
I20260809 16:29:33.988602 2142 ts_tablet_manager.cc:1918] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:29:33.988627 2137 ts_tablet_manager.cc:1939] T eed54efc0a4b4556874aabcce8d40485 P a2a6b20d23404b649a1a94d7d47a01c1: Deleting consensus metadata
I20260809 16:29:33.988732 2142 log.cc:1196] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/eed54efc0a4b4556874aabcce8d40485
I20260809 16:29:33.989048 2142 ts_tablet_manager.cc:1939] T eed54efc0a4b4556874aabcce8d40485 P e70fb780c0f243c1bcb8099b4492a7d5: Deleting consensus metadata
I20260809 16:29:33.989053 1195 catalog_manager.cc:4841] TS a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129:45619): tablet eed54efc0a4b4556874aabcce8d40485 (table test-table [id=36a4b880fe844934904ad1ad8d212ae4]) successfully deleted
I20260809 16:29:33.989517 1194 catalog_manager.cc:4841] TS e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130:46393): tablet eed54efc0a4b4556874aabcce8d40485 (table test-table [id=36a4b880fe844934904ad1ad8d212ae4]) successfully deleted
I20260809 16:29:34.012262 2075 raft_consensus.cc:482] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:29:34.012382 2075 raft_consensus.cc:504] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:34.012738 2075 leader_election.cc:288] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129:45619), e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130:46393)
I20260809 16:29:34.016566 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:34.012950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:46393 (local address 127.30.23.131:33019)
0809 16:29:34.013040 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:34.013052 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:34.013094 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:34.013385 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:34.013386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:34.013388 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:34.013459 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:34.013461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.014421 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.014422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:34.015555 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:34.015559 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.016303 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.016304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:34.016385 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:34.016388 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:29:34.016437 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:29:34.016476 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 16:29:34.017055 2065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:34.013172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:33019 (local address 127.30.23.130:46393)
0809 16:29:34.013212 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:34.013214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:34.013221 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:29:34.013258 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:34.013259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:34.013268 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:34.013315 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:34.013533 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.014307 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.015660 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.016177 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.016260 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:34.016269 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:34.016952 (+ 683us) server_negotiation.cc:299] Negotiation successful
0809 16:29:34.016989 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:29:34.017246 1710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57d127e7f7cc4755ae2af803e9a829a1" candidate_uuid: "31a2e03b656c4fc5ba79035b60086f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" is_pre_election: true
I20260809 16:29:34.017319 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:34.013194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:35269 (local address 127.30.23.129:45619)
0809 16:29:34.013238 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:34.013240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:34.013380 (+ 140us) server_negotiation.cc:407] Connection header received
0809 16:29:34.013481 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:34.013482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:34.013491 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:34.013521 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:34.014097 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.014884 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.016378 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.016940 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.016987 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:34.016994 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:34.017205 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 16:29:34.017239 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:29:34.017328 1710 raft_consensus.cc:2454] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a2e03b656c4fc5ba79035b60086f2c in term 0.
I20260809 16:29:34.017575 2148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:34.013039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45619 (local address 127.30.23.131:35269)
0809 16:29:34.013295 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:34.013304 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:34.013406 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:34.013583 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:34.013584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:34.013586 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:34.014001 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:34.014004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.015030 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.015031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:34.016278 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:34.016281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.017054 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.017055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:34.017126 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:34.017130 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:34.017455 (+ 325us) client_negotiation.cc:240] Negotiation successful
0809 16:29:34.017497 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":61,"threads_started":1}
I20260809 16:29:34.017877 1836 leader_election.cc:302] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a2e03b656c4fc5ba79035b60086f2c, e70fb780c0f243c1bcb8099b4492a7d5; no voters:
I20260809 16:29:34.017896 1472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57d127e7f7cc4755ae2af803e9a829a1" candidate_uuid: "31a2e03b656c4fc5ba79035b60086f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" is_pre_election: true
I20260809 16:29:34.017980 1472 raft_consensus.cc:2454] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a2e03b656c4fc5ba79035b60086f2c in term 0.
I20260809 16:29:34.018007 2075 raft_consensus.cc:2781] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:29:34.018040 2075 raft_consensus.cc:482] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:29:34.018060 2075 raft_consensus.cc:3037] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:34.019007 2075 raft_consensus.cc:504] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:34.019153 2075 leader_election.cc:288] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [CANDIDATE]: Term 1 election: Requested vote from peers a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129:45619), e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130:46393)
I20260809 16:29:34.019366 1710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57d127e7f7cc4755ae2af803e9a829a1" candidate_uuid: "31a2e03b656c4fc5ba79035b60086f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70fb780c0f243c1bcb8099b4492a7d5"
I20260809 16:29:34.019374 1472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57d127e7f7cc4755ae2af803e9a829a1" candidate_uuid: "31a2e03b656c4fc5ba79035b60086f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a6b20d23404b649a1a94d7d47a01c1"
I20260809 16:29:34.019430 1710 raft_consensus.cc:3037] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:34.019446 1472 raft_consensus.cc:3037] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:34.020162 1710 raft_consensus.cc:2454] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a2e03b656c4fc5ba79035b60086f2c in term 1.
I20260809 16:29:34.020262 1472 raft_consensus.cc:2454] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a2e03b656c4fc5ba79035b60086f2c in term 1.
I20260809 16:29:34.020330 1836 leader_election.cc:302] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a2e03b656c4fc5ba79035b60086f2c, e70fb780c0f243c1bcb8099b4492a7d5; no voters:
I20260809 16:29:34.020406 2075 raft_consensus.cc:2781] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:34.020535 2075 raft_consensus.cc:686] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 LEADER]: Becoming Leader. State: Replica: 31a2e03b656c4fc5ba79035b60086f2c, State: Running, Role: LEADER
I20260809 16:29:34.020602 2075 consensus_queue.cc:237] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:34.022006 1218 catalog_manager.cc:5495] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 31a2e03b656c4fc5ba79035b60086f2c (127.30.23.131). New cstate: current_term: 1 leader_uuid: "31a2e03b656c4fc5ba79035b60086f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } health_report { overall_health: HEALTHY } } }
I20260809 16:29:34.035416 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:34.032140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38888 (local address 127.30.23.131:41839)
0809 16:29:34.032179 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:34.032181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:34.032188 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:29:34.032222 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:34.032223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:34.032243 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:34.032317 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:34.032484 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.033218 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.034335 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.034631 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.034688 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:34.034895 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:34.035027 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:34.035091 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:34.035317 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 16:29:34.035338 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:29:34.041438 2065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:34.038326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:50435 (local address 127.30.23.130:46393)
0809 16:29:34.038369 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:34.038371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:34.038380 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:29:34.038412 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:34.038413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:34.038423 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:34.038497 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:34.038651 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.039435 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.040542 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.040860 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.040940 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:34.041141 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:34.041179 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:34.041239 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:34.041299 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 16:29:34.041320 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:29:34.041465 2039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:34.038242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:46393 (local address 127.30.23.131:50435)
0809 16:29:34.038322 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:34.038332 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:34.038368 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:34.038499 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:34.038500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:34.038501 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:29:34.038576 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:34.038578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.039541 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.039542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:34.040433 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:34.040436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.040950 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.040951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:34.041026 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:34.041044 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:29:34.041248 (+ 204us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:29:34.041250 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:29:34.041295 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:29:34.041381 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 16:29:34.044754 1472 raft_consensus.cc:1261] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Refusing update from remote peer 31a2e03b656c4fc5ba79035b60086f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:34.044799 1710 raft_consensus.cc:1261] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Refusing update from remote peer 31a2e03b656c4fc5ba79035b60086f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:34.045089 2149 consensus_queue.cc:1035] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:34.045202 2075 consensus_queue.cc:1035] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:34.054416 1798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:34.051078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:41839 (local address 127.30.23.130:56231)
0809 16:29:34.051162 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:34.051173 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:34.051212 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:34.051463 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:34.051464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:34.051466 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:29:34.051511 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:34.051513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.052564 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.052565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:34.053469 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:34.053473 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.053994 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.053995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:34.054057 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:34.054063 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:29:34.054240 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:29:34.054241 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:29:34.054288 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:29:34.054337 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 16:29:34.054405 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:34.051285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:56231 (local address 127.30.23.131:41839)
0809 16:29:34.051325 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:34.051327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:34.051333 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:29:34.051355 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:34.051356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:34.051366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:34.051414 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:34.051585 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.052345 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.053563 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:34.053877 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:34.053953 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:34.054138 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:34.054177 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:34.054219 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:34.054300 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 16:29:34.054329 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:29:34.054998 2109 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 16:29:35.582638 1219 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:35.582772 1219 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:35.584787 1219 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb: Sending DeleteTablet for 3 replicas of tablet 57d127e7f7cc4755ae2af803e9a829a1
I20260809 16:29:35.585116 1442 tablet_service.cc:1514] Processing DeleteTablet for tablet 57d127e7f7cc4755ae2af803e9a829a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:35 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:40970
I20260809 16:29:35.585117 1675 tablet_service.cc:1514] Processing DeleteTablet for tablet 57d127e7f7cc4755ae2af803e9a829a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:35 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:58794
I20260809 16:29:35.585165 1913 tablet_service.cc:1514] Processing DeleteTablet for tablet 57d127e7f7cc4755ae2af803e9a829a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:35 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:38882
I20260809 16:29:35.585711 2208 tablet_replica.cc:330] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1: stopping tablet replica
I20260809 16:29:35.585803 2208 raft_consensus.cc:2229] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:35.585862 2208 raft_consensus.cc:2258] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:35.586180 2208 ts_tablet_manager.cc:1905] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:35.588653 2208 ts_tablet_manager.cc:1918] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:29:35.588730 2208 log.cc:1196] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/wal/wals/57d127e7f7cc4755ae2af803e9a829a1
I20260809 16:29:35.588974 2208 ts_tablet_manager.cc:1939] T 57d127e7f7cc4755ae2af803e9a829a1 P a2a6b20d23404b649a1a94d7d47a01c1: Deleting consensus metadata
I20260809 16:29:35.589154 2211 tablet_replica.cc:330] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5: stopping tablet replica
I20260809 16:29:35.589232 2211 raft_consensus.cc:2229] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:35.589286 2211 raft_consensus.cc:2258] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:35.589439 2212 tablet_replica.cc:330] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c: stopping tablet replica
I20260809 16:29:35.589515 2212 raft_consensus.cc:2229] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:29:35.589627 2212 raft_consensus.cc:2258] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:35.589919 2212 ts_tablet_manager.cc:1905] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:35.590060 1195 catalog_manager.cc:4841] TS a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129:45619): tablet 57d127e7f7cc4755ae2af803e9a829a1 (table test-table [id=74afafd21683481fb2f7e83219050271]) successfully deleted
I20260809 16:29:35.590451 2211 ts_tablet_manager.cc:1905] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:35.591581 2212 ts_tablet_manager.cc:1918] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:29:35.591653 2212 log.cc:1196] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/57d127e7f7cc4755ae2af803e9a829a1
I20260809 16:29:35.591871 2212 ts_tablet_manager.cc:1939] T 57d127e7f7cc4755ae2af803e9a829a1 P 31a2e03b656c4fc5ba79035b60086f2c: Deleting consensus metadata
I20260809 16:29:35.592248 1194 catalog_manager.cc:4841] TS 31a2e03b656c4fc5ba79035b60086f2c (127.30.23.131:41839): tablet 57d127e7f7cc4755ae2af803e9a829a1 (table test-table [id=74afafd21683481fb2f7e83219050271]) successfully deleted
I20260809 16:29:35.593482 2211 ts_tablet_manager.cc:1918] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:29:35.593536 2211 log.cc:1196] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/57d127e7f7cc4755ae2af803e9a829a1
I20260809 16:29:35.593750 2211 ts_tablet_manager.cc:1939] T 57d127e7f7cc4755ae2af803e9a829a1 P e70fb780c0f243c1bcb8099b4492a7d5: Deleting consensus metadata
I20260809 16:29:35.594112 1194 catalog_manager.cc:4841] TS e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130:46393): tablet 57d127e7f7cc4755ae2af803e9a829a1 (table test-table [id=74afafd21683481fb2f7e83219050271]) successfully deleted
Aug 09 16:29:35 dist-test-slave-j2vl krb5kdc[1140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292975, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:29:35 dist-test-slave-j2vl krb5kdc[1140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292975, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:35.614238 2227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:35.606290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40880 (local address 127.30.23.190:34131)
0809 16:29:35.606464 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:35.606466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:35.606475 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:29:35.606503 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:35.606504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:35.606515 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:35.606580 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:35.607194 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:35.607704 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:35.608514 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:35.608765 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:35.608833 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:35.611424 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:35.611426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:35.611426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:35.611434 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:35.612910 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:35.613414 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:35.613416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:35.613416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:35.613489 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:35.613708 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:35.613709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:35.613709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:35.613824 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:35.613913 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:35.614002 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 16:29:35.614108 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
I20260809 16:29:35.615948 1219 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:29:35.616127 1219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:29:35.617674 1675 tablet_service.cc:1467] Processing CreateTablet for tablet 7286de381b1d4d7b8c58103f386d6366 (DEFAULT_TABLE table=test-table [id=6afcc458a6b948af9644e9b4f821d46e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:35.617694 1913 tablet_service.cc:1467] Processing CreateTablet for tablet 7286de381b1d4d7b8c58103f386d6366 (DEFAULT_TABLE table=test-table [id=6afcc458a6b948af9644e9b4f821d46e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:35.617839 1675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7286de381b1d4d7b8c58103f386d6366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:35.617844 1913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7286de381b1d4d7b8c58103f386d6366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:35.618908 2228 tablet_bootstrap.cc:492] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c: Bootstrap starting.
I20260809 16:29:35.619055 1442 tablet_service.cc:1467] Processing CreateTablet for tablet 7286de381b1d4d7b8c58103f386d6366 (DEFAULT_TABLE table=test-table [id=6afcc458a6b948af9644e9b4f821d46e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:35.619210 1442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7286de381b1d4d7b8c58103f386d6366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:35.619174 2230 tablet_bootstrap.cc:492] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5: Bootstrap starting.
I20260809 16:29:35.619529 2228 tablet_bootstrap.cc:654] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:35.619673 2230 tablet_bootstrap.cc:654] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:35.620236 2228 tablet_bootstrap.cc:492] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c: No bootstrap required, opened a new log
I20260809 16:29:35.620278 2228 ts_tablet_manager.cc:1397] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:35.620462 2228 raft_consensus.cc:348] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.620541 2228 raft_consensus.cc:374] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:35.620556 2228 raft_consensus.cc:729] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a2e03b656c4fc5ba79035b60086f2c, State: Initialized, Role: FOLLOWER
I20260809 16:29:35.620590 2233 tablet_bootstrap.cc:492] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1: Bootstrap starting.
I20260809 16:29:35.620636 2228 consensus_queue.cc:260] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.620766 2228 ts_tablet_manager.cc:1428] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:29:35.621136 2233 tablet_bootstrap.cc:654] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:35.621245 2230 tablet_bootstrap.cc:492] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5: No bootstrap required, opened a new log
I20260809 16:29:35.621304 2230 ts_tablet_manager.cc:1397] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:29:35.621457 2230 raft_consensus.cc:348] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.621532 2230 raft_consensus.cc:374] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:35.621558 2230 raft_consensus.cc:729] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70fb780c0f243c1bcb8099b4492a7d5, State: Initialized, Role: FOLLOWER
I20260809 16:29:35.621624 2230 consensus_queue.cc:260] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.621726 2230 ts_tablet_manager.cc:1428] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:35.622038 2233 tablet_bootstrap.cc:492] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1: No bootstrap required, opened a new log
I20260809 16:29:35.622097 2233 ts_tablet_manager.cc:1397] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:29:35.622222 2233 raft_consensus.cc:348] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.622303 2233 raft_consensus.cc:374] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:35.622330 2233 raft_consensus.cc:729] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a6b20d23404b649a1a94d7d47a01c1, State: Initialized, Role: FOLLOWER
I20260809 16:29:35.622408 2233 consensus_queue.cc:260] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.622607 2233 ts_tablet_manager.cc:1428] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:29:35.775905 2196 raft_consensus.cc:482] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:29:35.776024 2196 raft_consensus.cc:504] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.776238 2196 leader_election.cc:288] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129:45619), e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130:46393)
I20260809 16:29:35.776577 1472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7286de381b1d4d7b8c58103f386d6366" candidate_uuid: "31a2e03b656c4fc5ba79035b60086f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" is_pre_election: true
I20260809 16:29:35.776577 1710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7286de381b1d4d7b8c58103f386d6366" candidate_uuid: "31a2e03b656c4fc5ba79035b60086f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" is_pre_election: true
I20260809 16:29:35.776695 1472 raft_consensus.cc:2454] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a2e03b656c4fc5ba79035b60086f2c in term 0.
I20260809 16:29:35.776695 1710 raft_consensus.cc:2454] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a2e03b656c4fc5ba79035b60086f2c in term 0.
I20260809 16:29:35.776887 1837 leader_election.cc:302] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a2e03b656c4fc5ba79035b60086f2c, a2a6b20d23404b649a1a94d7d47a01c1; no voters:
I20260809 16:29:35.777022 2196 raft_consensus.cc:2781] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:29:35.777053 2196 raft_consensus.cc:482] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:29:35.777067 2196 raft_consensus.cc:3037] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:35.777834 2196 raft_consensus.cc:504] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.777983 2196 leader_election.cc:288] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [CANDIDATE]: Term 1 election: Requested vote from peers a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129:45619), e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130:46393)
I20260809 16:29:35.778162 1710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7286de381b1d4d7b8c58103f386d6366" candidate_uuid: "31a2e03b656c4fc5ba79035b60086f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70fb780c0f243c1bcb8099b4492a7d5"
I20260809 16:29:35.778231 1710 raft_consensus.cc:3037] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:35.778198 1472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7286de381b1d4d7b8c58103f386d6366" candidate_uuid: "31a2e03b656c4fc5ba79035b60086f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a6b20d23404b649a1a94d7d47a01c1"
I20260809 16:29:35.778311 1472 raft_consensus.cc:3037] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:35.778901 1710 raft_consensus.cc:2454] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a2e03b656c4fc5ba79035b60086f2c in term 1.
I20260809 16:29:35.779021 1472 raft_consensus.cc:2454] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a2e03b656c4fc5ba79035b60086f2c in term 1.
I20260809 16:29:35.779083 1836 leader_election.cc:302] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a2e03b656c4fc5ba79035b60086f2c, e70fb780c0f243c1bcb8099b4492a7d5; no voters:
I20260809 16:29:35.779176 2196 raft_consensus.cc:2781] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:35.779228 2196 raft_consensus.cc:686] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 LEADER]: Becoming Leader. State: Replica: 31a2e03b656c4fc5ba79035b60086f2c, State: Running, Role: LEADER
I20260809 16:29:35.779286 2196 consensus_queue.cc:237] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.780009 1218 catalog_manager.cc:5495] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 31a2e03b656c4fc5ba79035b60086f2c (127.30.23.131). New cstate: current_term: 1 leader_uuid: "31a2e03b656c4fc5ba79035b60086f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } health_report { overall_health: HEALTHY } } }
I20260809 16:29:35.808910 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:35.806249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51676 (local address 127.30.23.131:41839)
0809 16:29:35.806485 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:35.806487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:35.806494 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:29:35.806516 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:35.806517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:35.806527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:35.806612 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:35.806785 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:35.807293 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:35.808088 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:35.808335 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:35.808434 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:35.808596 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:35.808640 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:35.808697 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:35.808784 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 16:29:35.808809 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":118,"threads_started":1}
I20260809 16:29:35.811750 1710 raft_consensus.cc:1261] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Refusing update from remote peer 31a2e03b656c4fc5ba79035b60086f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:35.811872 1472 raft_consensus.cc:1261] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Refusing update from remote peer 31a2e03b656c4fc5ba79035b60086f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:35.811995 2196 consensus_queue.cc:1035] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:35.812215 2196 consensus_queue.cc:1035] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:35.822711 1219 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:35.823009 1219 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:35.823772 1219 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb: Sending DeleteTablet for 3 replicas of tablet 7286de381b1d4d7b8c58103f386d6366
I20260809 16:29:35.824103 1913 tablet_service.cc:1514] Processing DeleteTablet for tablet 7286de381b1d4d7b8c58103f386d6366 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:35 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:38882
I20260809 16:29:35.824105 1675 tablet_service.cc:1514] Processing DeleteTablet for tablet 7286de381b1d4d7b8c58103f386d6366 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:35 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:58794
I20260809 16:29:35.824122 1442 tablet_service.cc:1514] Processing DeleteTablet for tablet 7286de381b1d4d7b8c58103f386d6366 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:35 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:40970
I20260809 16:29:35.824205 2211 tablet_replica.cc:330] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5: stopping tablet replica
I20260809 16:29:35.824222 2208 tablet_replica.cc:330] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1: stopping tablet replica
I20260809 16:29:35.824266 2211 raft_consensus.cc:2229] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:35.824302 2211 raft_consensus.cc:2258] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:35.824299 2208 raft_consensus.cc:2229] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:35.824357 2208 raft_consensus.cc:2258] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:35.824376 2212 tablet_replica.cc:330] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c: stopping tablet replica
I20260809 16:29:35.824380 1219 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 16:29:35.824440 2212 raft_consensus.cc:2229] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 LEADER]: Raft consensus shutting down.
W20260809 16:29:35.824509 1219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:29:35.824522 2212 raft_consensus.cc:2258] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:35.824702 2211 ts_tablet_manager.cc:1905] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:35.825598 2208 ts_tablet_manager.cc:1905] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:35.825850 1675 tablet_service.cc:1467] Processing CreateTablet for tablet aedfc96065ce4257be8fdfdce23569f2 (DEFAULT_TABLE table=test-table [id=aa3452b3e5d94a9588e884eccbfe0ed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:35.826059 1675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aedfc96065ce4257be8fdfdce23569f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:35.826103 1913 tablet_service.cc:1467] Processing CreateTablet for tablet aedfc96065ce4257be8fdfdce23569f2 (DEFAULT_TABLE table=test-table [id=aa3452b3e5d94a9588e884eccbfe0ed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:35.826218 1913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aedfc96065ce4257be8fdfdce23569f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:35.826747 2211 ts_tablet_manager.cc:1918] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:29:35.826803 2211 log.cc:1196] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/7286de381b1d4d7b8c58103f386d6366
I20260809 16:29:35.827067 2211 ts_tablet_manager.cc:1939] T 7286de381b1d4d7b8c58103f386d6366 P e70fb780c0f243c1bcb8099b4492a7d5: Deleting consensus metadata
I20260809 16:29:35.827564 2208 ts_tablet_manager.cc:1918] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:29:35.827592 1194 catalog_manager.cc:4841] TS e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130:46393): tablet 7286de381b1d4d7b8c58103f386d6366 (table test-table [id=6afcc458a6b948af9644e9b4f821d46e]) successfully deleted
I20260809 16:29:35.827621 2208 log.cc:1196] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-0/wal/wals/7286de381b1d4d7b8c58103f386d6366
I20260809 16:29:35.827674 2228 tablet_bootstrap.cc:492] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c: Bootstrap starting.
I20260809 16:29:35.827847 2208 ts_tablet_manager.cc:1939] T 7286de381b1d4d7b8c58103f386d6366 P a2a6b20d23404b649a1a94d7d47a01c1: Deleting consensus metadata
I20260809 16:29:35.828042 2228 tablet_bootstrap.cc:654] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:35.828092 2230 tablet_bootstrap.cc:492] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5: Bootstrap starting.
I20260809 16:29:35.828357 2230 tablet_bootstrap.cc:654] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:35.828647 2228 tablet_bootstrap.cc:492] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c: No bootstrap required, opened a new log
I20260809 16:29:35.828718 2228 ts_tablet_manager.cc:1397] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:35.828743 2212 ts_tablet_manager.cc:1905] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:35.828919 2228 raft_consensus.cc:348] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.828990 2228 raft_consensus.cc:374] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:35.829005 2228 raft_consensus.cc:729] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a2e03b656c4fc5ba79035b60086f2c, State: Initialized, Role: FOLLOWER
I20260809 16:29:35.829035 2230 tablet_bootstrap.cc:492] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5: No bootstrap required, opened a new log
I20260809 16:29:35.829075 2230 ts_tablet_manager.cc:1397] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:29:35.829063 2228 consensus_queue.cc:260] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.829161 2228 ts_tablet_manager.cc:1428] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:35.829169 2230 raft_consensus.cc:348] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.829226 2230 raft_consensus.cc:374] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:35.829246 2230 raft_consensus.cc:729] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70fb780c0f243c1bcb8099b4492a7d5, State: Initialized, Role: FOLLOWER
I20260809 16:29:35.829298 2230 consensus_queue.cc:260] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.829423 2230 ts_tablet_manager.cc:1428] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:35.829803 1195 catalog_manager.cc:4841] TS a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129:45619): tablet 7286de381b1d4d7b8c58103f386d6366 (table test-table [id=6afcc458a6b948af9644e9b4f821d46e]) successfully deleted
I20260809 16:29:35.829836 1442 tablet_service.cc:1467] Processing CreateTablet for tablet aedfc96065ce4257be8fdfdce23569f2 (DEFAULT_TABLE table=test-table [id=aa3452b3e5d94a9588e884eccbfe0ed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:35.829959 1442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aedfc96065ce4257be8fdfdce23569f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:35.830287 2212 ts_tablet_manager.cc:1918] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:29:35.830371 2212 log.cc:1196] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/7286de381b1d4d7b8c58103f386d6366
I20260809 16:29:35.830574 2212 ts_tablet_manager.cc:1939] T 7286de381b1d4d7b8c58103f386d6366 P 31a2e03b656c4fc5ba79035b60086f2c: Deleting consensus metadata
I20260809 16:29:35.830945 1194 catalog_manager.cc:4841] TS 31a2e03b656c4fc5ba79035b60086f2c (127.30.23.131:41839): tablet 7286de381b1d4d7b8c58103f386d6366 (table test-table [id=6afcc458a6b948af9644e9b4f821d46e]) successfully deleted
I20260809 16:29:35.831128 2233 tablet_bootstrap.cc:492] T aedfc96065ce4257be8fdfdce23569f2 P a2a6b20d23404b649a1a94d7d47a01c1: Bootstrap starting.
I20260809 16:29:35.831499 2233 tablet_bootstrap.cc:654] T aedfc96065ce4257be8fdfdce23569f2 P a2a6b20d23404b649a1a94d7d47a01c1: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:35.832113 2233 tablet_bootstrap.cc:492] T aedfc96065ce4257be8fdfdce23569f2 P a2a6b20d23404b649a1a94d7d47a01c1: No bootstrap required, opened a new log
I20260809 16:29:35.832204 2233 ts_tablet_manager.cc:1397] T aedfc96065ce4257be8fdfdce23569f2 P a2a6b20d23404b649a1a94d7d47a01c1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:35.832327 2233 raft_consensus.cc:348] T aedfc96065ce4257be8fdfdce23569f2 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.832405 2233 raft_consensus.cc:374] T aedfc96065ce4257be8fdfdce23569f2 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:35.832432 2233 raft_consensus.cc:729] T aedfc96065ce4257be8fdfdce23569f2 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a6b20d23404b649a1a94d7d47a01c1, State: Initialized, Role: FOLLOWER
I20260809 16:29:35.832495 2233 consensus_queue.cc:260] T aedfc96065ce4257be8fdfdce23569f2 P a2a6b20d23404b649a1a94d7d47a01c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:35.832613 2233 ts_tablet_manager.cc:1428] T aedfc96065ce4257be8fdfdce23569f2 P a2a6b20d23404b649a1a94d7d47a01c1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:36.143289 2149 raft_consensus.cc:482] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:29:36.143453 2149 raft_consensus.cc:504] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:36.143683 2149 leader_election.cc:288] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129:45619), e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130:46393)
I20260809 16:29:36.144030 1710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aedfc96065ce4257be8fdfdce23569f2" candidate_uuid: "31a2e03b656c4fc5ba79035b60086f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" is_pre_election: true
I20260809 16:29:36.144253 1710 raft_consensus.cc:2454] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a2e03b656c4fc5ba79035b60086f2c in term 0.
I20260809 16:29:36.144213 1472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aedfc96065ce4257be8fdfdce23569f2" candidate_uuid: "31a2e03b656c4fc5ba79035b60086f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" is_pre_election: true
I20260809 16:29:36.144320 1472 raft_consensus.cc:2454] T aedfc96065ce4257be8fdfdce23569f2 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31a2e03b656c4fc5ba79035b60086f2c in term 0.
I20260809 16:29:36.144642 1836 leader_election.cc:302] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a2e03b656c4fc5ba79035b60086f2c, e70fb780c0f243c1bcb8099b4492a7d5; no voters:
I20260809 16:29:36.144775 2149 raft_consensus.cc:2781] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:29:36.144840 2149 raft_consensus.cc:482] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:29:36.144886 2149 raft_consensus.cc:3037] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:36.145766 2149 raft_consensus.cc:504] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:36.145920 2149 leader_election.cc:288] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [CANDIDATE]: Term 1 election: Requested vote from peers a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129:45619), e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130:46393)
I20260809 16:29:36.146109 1472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aedfc96065ce4257be8fdfdce23569f2" candidate_uuid: "31a2e03b656c4fc5ba79035b60086f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a6b20d23404b649a1a94d7d47a01c1"
I20260809 16:29:36.146108 1710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aedfc96065ce4257be8fdfdce23569f2" candidate_uuid: "31a2e03b656c4fc5ba79035b60086f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70fb780c0f243c1bcb8099b4492a7d5"
I20260809 16:29:36.146193 1472 raft_consensus.cc:3037] T aedfc96065ce4257be8fdfdce23569f2 P a2a6b20d23404b649a1a94d7d47a01c1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:36.146194 1710 raft_consensus.cc:3037] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:36.146972 1472 raft_consensus.cc:2454] T aedfc96065ce4257be8fdfdce23569f2 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a2e03b656c4fc5ba79035b60086f2c in term 1.
I20260809 16:29:36.146981 1710 raft_consensus.cc:2454] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31a2e03b656c4fc5ba79035b60086f2c in term 1.
I20260809 16:29:36.147190 1836 leader_election.cc:302] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31a2e03b656c4fc5ba79035b60086f2c, e70fb780c0f243c1bcb8099b4492a7d5; no voters:
I20260809 16:29:36.147334 2149 raft_consensus.cc:2781] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:36.147415 2149 raft_consensus.cc:686] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 LEADER]: Becoming Leader. State: Replica: 31a2e03b656c4fc5ba79035b60086f2c, State: Running, Role: LEADER
I20260809 16:29:36.147473 2149 consensus_queue.cc:237] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } }
I20260809 16:29:36.148141 1218 catalog_manager.cc:5495] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 31a2e03b656c4fc5ba79035b60086f2c (127.30.23.131). New cstate: current_term: 1 leader_uuid: "31a2e03b656c4fc5ba79035b60086f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31a2e03b656c4fc5ba79035b60086f2c" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41839 } health_report { overall_health: HEALTHY } } }
I20260809 16:29:36.190516 1710 raft_consensus.cc:1261] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Refusing update from remote peer 31a2e03b656c4fc5ba79035b60086f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:36.190521 1472 raft_consensus.cc:1261] T aedfc96065ce4257be8fdfdce23569f2 P a2a6b20d23404b649a1a94d7d47a01c1 [term 1 FOLLOWER]: Refusing update from remote peer 31a2e03b656c4fc5ba79035b60086f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:36.190752 2196 consensus_queue.cc:1035] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e70fb780c0f243c1bcb8099b4492a7d5" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:36.190898 2149 consensus_queue.cc:1035] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2a6b20d23404b649a1a94d7d47a01c1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 16:29:36.195930 2170 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 16:29:37.716775 1217 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:37.716915 1217 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:37.717738 1217 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25f744ade59a44e7be47378205a599bb: Sending DeleteTablet for 3 replicas of tablet aedfc96065ce4257be8fdfdce23569f2
I20260809 16:29:37.718035 1913 tablet_service.cc:1514] Processing DeleteTablet for tablet aedfc96065ce4257be8fdfdce23569f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:37 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:38882
I20260809 16:29:37.718050 1675 tablet_service.cc:1514] Processing DeleteTablet for tablet aedfc96065ce4257be8fdfdce23569f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:37 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:58794
I20260809 16:29:37.718190 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 1341
I20260809 16:29:37.723805 2296 tablet_replica.cc:330] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5: stopping tablet replica
I20260809 16:29:37.723912 2296 raft_consensus.cc:2229] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:37.723971 2296 raft_consensus.cc:2258] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:37.724280 2296 ts_tablet_manager.cc:1905] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:37.726271 2296 ts_tablet_manager.cc:1918] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:29:37.726351 2296 log.cc:1196] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/aedfc96065ce4257be8fdfdce23569f2
I20260809 16:29:37.726621 2296 ts_tablet_manager.cc:1939] T aedfc96065ce4257be8fdfdce23569f2 P e70fb780c0f243c1bcb8099b4492a7d5: Deleting consensus metadata
I20260809 16:29:37.727137 1194 catalog_manager.cc:4841] TS e70fb780c0f243c1bcb8099b4492a7d5 (127.30.23.130:46393): tablet aedfc96065ce4257be8fdfdce23569f2 (table test-table [id=aa3452b3e5d94a9588e884eccbfe0ed6]) successfully deleted
I20260809 16:29:37.731707 2294 tablet_replica.cc:330] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c: stopping tablet replica
I20260809 16:29:37.731792 2294 raft_consensus.cc:2229] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:29:37.731931 2294 raft_consensus.cc:2258] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:37.732216 2294 ts_tablet_manager.cc:1905] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:37.733873 2294 ts_tablet_manager.cc:1918] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:29:37.733956 2294 log.cc:1196] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/aedfc96065ce4257be8fdfdce23569f2
I20260809 16:29:37.734328 2294 ts_tablet_manager.cc:1939] T aedfc96065ce4257be8fdfdce23569f2 P 31a2e03b656c4fc5ba79035b60086f2c: Deleting consensus metadata
I20260809 16:29:37.734735 1194 catalog_manager.cc:4841] TS 31a2e03b656c4fc5ba79035b60086f2c (127.30.23.131:41839): tablet aedfc96065ce4257be8fdfdce23569f2 (table test-table [id=aa3452b3e5d94a9588e884eccbfe0ed6]) successfully deleted
W20260809 16:29:37.738131 1195 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:45619 (error 108)
I20260809 16:29:37.738451 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 1562
I20260809 16:29:37.738875 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:37.738642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45619 (local address 127.0.0.1:34410)
0809 16:29:37.738781 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:37.738827 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:45619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
W20260809 16:29:37.738996 1195 catalog_manager.cc:4584] TS a2a6b20d23404b649a1a94d7d47a01c1 (127.30.23.129:45619): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aedfc96065ce4257be8fdfdce23569f2: Network error: Client connection negotiation failed: client connection to 127.30.23.129:45619: connect: Connection refused (error 111)
I20260809 16:29:37.756629 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 1805
I20260809 16:29:37.769488 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 1164
2026-08-09T16:29:37Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6184 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:29:39 dist-test-slave-j2vl krb5kdc[2303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:29:39 dist-test-slave-j2vl krb5kdc[2303](info): set up 2 sockets
Aug 09 16:29:39 dist-test-slave-j2vl krb5kdc[2303](info): commencing operation
krb5kdc: starting...
W20260809 16:29:40.464270 30814 mini_kdc.cc:121] Time spent starting KDC: real 2.659s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:29:40 dist-test-slave-j2vl krb5kdc[2303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:29:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:29:40.609210 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:34479
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:42081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:34479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:40.698027 2325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:40.698304 2325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:40.698346 2325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:40.699509 2325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:40.699579 2325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:40.699604 2325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:40.699632 2325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:40.699666 2325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:40.701184 2325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:42081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:34479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:34479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:40.701495 2325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:40.701781 2325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:40.707454 2334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:40.707449 2338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:40.707454 2335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:40.707736 2325 server_base.cc:1034] running on GCE node
I20260809 16:29:40.708007 2325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:40.708343 2325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:40.709522 2325 hybrid_clock.cc:648] HybridClock initialized: now 1786292980709497 us; error 39 us; skew 500 ppm
Aug 09 16:29:40 dist-test-slave-j2vl krb5kdc[2303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292980, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:40.712361 2325 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:29:40.712867 2325 webserver.cc:460] Webserver started at http://127.30.23.190:40139/ using document root <none> and password file <none>
I20260809 16:29:40.713084 2325 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:40.713127 2325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:40.713222 2325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:40.714352 2325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "27ddebd6ba5b4d80a07983b6906c5238"
format_stamp: "Formatted at 2026-08-09 16:29:40 on dist-test-slave-j2vl"
server_key: "b7887dec2710aa2bdc149ef024bcec31"
server_key_iv: "6bd6d483816bc1975d0dd6727bd92016"
server_key_version: "encryptionkey@0"
I20260809 16:29:40.714664 2325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "27ddebd6ba5b4d80a07983b6906c5238"
format_stamp: "Formatted at 2026-08-09 16:29:40 on dist-test-slave-j2vl"
server_key: "b7887dec2710aa2bdc149ef024bcec31"
server_key_iv: "6bd6d483816bc1975d0dd6727bd92016"
server_key_version: "encryptionkey@0"
I20260809 16:29:40.716084 2325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:29:40.716944 2352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:40.717141 2325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:40.717234 2325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "27ddebd6ba5b4d80a07983b6906c5238"
format_stamp: "Formatted at 2026-08-09 16:29:40 on dist-test-slave-j2vl"
server_key: "b7887dec2710aa2bdc149ef024bcec31"
server_key_iv: "6bd6d483816bc1975d0dd6727bd92016"
server_key_version: "encryptionkey@0"
I20260809 16:29:40.717294 2325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:40.745066 2325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:40.747488 2325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:40.747604 2325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:40.751739 2325 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:34479
I20260809 16:29:40.751798 2432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:34479 every 8 connection(s)
I20260809 16:29:40.752223 2325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:40.753161 2433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:40.755581 2433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238: Bootstrap starting.
I20260809 16:29:40.756031 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 2325
I20260809 16:29:40.756137 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:40.756326 30814 external_mini_cluster.cc:1428] Setting key 9da257c60d3a8001f63eb4da0e96c61b
I20260809 16:29:40.756387 2433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:40.756744 2433 log.cc:824] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:40.757712 2433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238: No bootstrap required, opened a new log
I20260809 16:29:40.759575 2433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ddebd6ba5b4d80a07983b6906c5238" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34479 } }
I20260809 16:29:40.759739 2433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:40.759793 2433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27ddebd6ba5b4d80a07983b6906c5238, State: Initialized, Role: FOLLOWER
I20260809 16:29:40.759935 2433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ddebd6ba5b4d80a07983b6906c5238" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34479 } }
I20260809 16:29:40.760025 2433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:40.760070 2433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:40.760119 2433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:40.760756 2433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ddebd6ba5b4d80a07983b6906c5238" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34479 } }
I20260809 16:29:40.760881 2433 leader_election.cc:302] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27ddebd6ba5b4d80a07983b6906c5238; no voters:
I20260809 16:29:40.761087 2433 leader_election.cc:288] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:40.761157 2445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:40.761394 2445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [term 1 LEADER]: Becoming Leader. State: Replica: 27ddebd6ba5b4d80a07983b6906c5238, State: Running, Role: LEADER
I20260809 16:29:40.761471 2433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:40.761554 2445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ddebd6ba5b4d80a07983b6906c5238" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34479 } }
I20260809 16:29:40.761875 2445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27ddebd6ba5b4d80a07983b6906c5238. Latest consensus state: current_term: 1 leader_uuid: "27ddebd6ba5b4d80a07983b6906c5238" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ddebd6ba5b4d80a07983b6906c5238" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34479 } } }
I20260809 16:29:40.761912 2446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27ddebd6ba5b4d80a07983b6906c5238" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ddebd6ba5b4d80a07983b6906c5238" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34479 } } }
I20260809 16:29:40.762044 2445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:40.762106 2446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:40.762526 2457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:40.763043 2457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:40.764781 2457 catalog_manager.cc:1347] Generated new cluster ID: fc3701203f7d42228adf28b44ae599ae
I20260809 16:29:40.764842 2457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:29:40 dist-test-slave-j2vl krb5kdc[2303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:40.768925 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:40.756909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42628 (local address 127.30.23.190:34479)
0809 16:29:40.757186 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:40.757196 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:40.757231 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:29:40.757810 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:40.757819 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:40.757880 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:40.758033 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:40.758670 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:40.759564 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:40.760545 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:40.761051 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:40.761188 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:40.765671 (+ 4483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:40.765675 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:40.765675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:40.765687 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:40.767407 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:40.767790 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:40.767793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:40.767793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:40.767890 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:40.768105 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:40.768106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:40.768107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:40.768354 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:40.768525 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:40.768623 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 16:29:40.768763 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":103,"threads_started":1}
I20260809 16:29:40.798655 2457 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:40.799347 2457 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:40.804888 2457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27ddebd6ba5b4d80a07983b6906c5238: Generated new TSK 0
I20260809 16:29:40.805162 2457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:29:40.841550 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34479
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:42081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:40.926820 2478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:40.927049 2478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:40.927073 2478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:40.928293 2478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:40.928347 2478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:40.928375 2478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:29:40.929953 2478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:42081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:40.930234 2478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:40.930497 2478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:40.935963 2489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:40.935956 2485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:40.936192 2478 server_base.cc:1034] running on GCE node
W20260809 16:29:40.935956 2486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:40.936652 2478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:40.936980 2478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:40.938165 2478 hybrid_clock.cc:648] HybridClock initialized: now 1786292980938140 us; error 31 us; skew 500 ppm
Aug 09 16:29:40 dist-test-slave-j2vl krb5kdc[2303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292980, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:40.941030 2478 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:29:40.941627 2478 webserver.cc:460] Webserver started at http://127.30.23.129:37209/ using document root <none> and password file <none>
I20260809 16:29:40.941897 2478 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:40.941968 2478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:40.942126 2478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:40.943225 2478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "2d273b8c983f46c9a6403d2a45a1c2ea"
format_stamp: "Formatted at 2026-08-09 16:29:40 on dist-test-slave-j2vl"
server_key: "b9846545d606591f62ee8a235bf99445"
server_key_iv: "be4f30e431510b0336835310705e4952"
server_key_version: "encryptionkey@0"
I20260809 16:29:40.943599 2478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "2d273b8c983f46c9a6403d2a45a1c2ea"
format_stamp: "Formatted at 2026-08-09 16:29:40 on dist-test-slave-j2vl"
server_key: "b9846545d606591f62ee8a235bf99445"
server_key_iv: "be4f30e431510b0336835310705e4952"
server_key_version: "encryptionkey@0"
I20260809 16:29:40.945019 2478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:29:40.945945 2504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:40.946168 2478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:29:40.946327 2478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "2d273b8c983f46c9a6403d2a45a1c2ea"
format_stamp: "Formatted at 2026-08-09 16:29:40 on dist-test-slave-j2vl"
server_key: "b9846545d606591f62ee8a235bf99445"
server_key_iv: "be4f30e431510b0336835310705e4952"
server_key_version: "encryptionkey@0"
I20260809 16:29:40.946412 2478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:40.957157 2478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:40.959975 2478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:40.960109 2478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:40.960316 2478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:40.960770 2478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:40.960870 2478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:40.960927 2478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:40.960974 2478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:40.968158 2478 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:39377
I20260809 16:29:40.968307 2679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:39377 every 8 connection(s)
I20260809 16:29:40.968636 2478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:29:40 dist-test-slave-j2vl krb5kdc[2303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292980, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:40.977742 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 2478
I20260809 16:29:40.977850 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:29:40.978063 30814 external_mini_cluster.cc:1428] Setting key 93ae4f6ffc2c733548c4a00971d3be6f
I20260809 16:29:40.978842 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:40.969576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:45221 (local address 127.30.23.190:34479)
0809 16:29:40.969648 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:40.969651 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:40.969859 (+ 208us) server_negotiation.cc:407] Connection header received
0809 16:29:40.970354 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:40.970356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:40.970368 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:40.970446 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:40.971499 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:40.972042 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:40.973090 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:40.973316 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:40.973453 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:40.975710 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:40.975711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:40.975712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:40.975720 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:40.977152 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:40.977699 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:40.977700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:40.977700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:40.977760 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:40.978087 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:40.978088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:40.978088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:40.978246 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:40.978304 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:40.978588 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 16:29:40.978710 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 16:29:40.978989 2682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:40.969528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34479 (local address 127.30.23.129:45221)
0809 16:29:40.969694 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:40.969739 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:40.970238 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:40.970570 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:40.970574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:40.971164 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:40.971376 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:40.971381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:40.972148 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:40.972149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:40.972981 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:40.972984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:40.973475 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:40.973476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:40.973585 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:40.974094 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:40.974103 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:40.975604 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:40.977256 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:40.977264 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:40.977266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:40.977619 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:40.977867 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:40.977867 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:40.977868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:40.978011 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:40.978301 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:40.978308 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:40.978461 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 16:29:40.978549 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:29:40.978723 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260809 16:29:40.980826 2680 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34479
I20260809 16:29:40.980926 2680 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:40.981114 2680 heartbeater.cc:511] Master 127.30.23.190:34479 requested a full tablet report, sending...
I20260809 16:29:40.981738 2382 ts_manager.cc:194] Registered new tserver with Master: 2d273b8c983f46c9a6403d2a45a1c2ea (127.30.23.129:39377)
I20260809 16:29:40.982658 2382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:45221
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:29:41.009265 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34479
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:42081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:41.099193 2687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:41.099435 2687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:41.099462 2687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:41.100658 2687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:41.100710 2687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:41.100739 2687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:29:41.102244 2687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:42081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:41.102465 2687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:41.102665 2687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:29:41.108371 2687 server_base.cc:1034] running on GCE node
W20260809 16:29:41.108323 2694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:41.108307 2696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:41.108317 2698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:41.108821 2687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:41.109139 2687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:41.110348 2687 hybrid_clock.cc:648] HybridClock initialized: now 1786292981110317 us; error 57 us; skew 500 ppm
Aug 09 16:29:41 dist-test-slave-j2vl krb5kdc[2303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292981, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:41.113144 2687 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:29:41.113747 2687 webserver.cc:460] Webserver started at http://127.30.23.130:37371/ using document root <none> and password file <none>
I20260809 16:29:41.114002 2687 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:41.114048 2687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:41.114146 2687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:41.115306 2687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "d0dc1da0ac6645628b409ed159152b8c"
format_stamp: "Formatted at 2026-08-09 16:29:41 on dist-test-slave-j2vl"
server_key: "387d1ce78b8f06c7056828ba9086b1e6"
server_key_iv: "8f7e8933b2e67c4fa11c108f0b129699"
server_key_version: "encryptionkey@0"
I20260809 16:29:41.115633 2687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "d0dc1da0ac6645628b409ed159152b8c"
format_stamp: "Formatted at 2026-08-09 16:29:41 on dist-test-slave-j2vl"
server_key: "387d1ce78b8f06c7056828ba9086b1e6"
server_key_iv: "8f7e8933b2e67c4fa11c108f0b129699"
server_key_version: "encryptionkey@0"
I20260809 16:29:41.116989 2687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:29:41.117790 2712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:41.118033 2687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:29:41.118114 2687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "d0dc1da0ac6645628b409ed159152b8c"
format_stamp: "Formatted at 2026-08-09 16:29:41 on dist-test-slave-j2vl"
server_key: "387d1ce78b8f06c7056828ba9086b1e6"
server_key_iv: "8f7e8933b2e67c4fa11c108f0b129699"
server_key_version: "encryptionkey@0"
I20260809 16:29:41.118170 2687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:41.136315 2687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:41.138773 2687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:41.138900 2687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:41.139137 2687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:41.139501 2687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:41.139541 2687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:41.139564 2687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:41.139577 2687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:41.146770 2687 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:44781
I20260809 16:29:41.146804 2885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:44781 every 8 connection(s)
I20260809 16:29:41.147184 2687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:29:41 dist-test-slave-j2vl krb5kdc[2303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292981, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:41.156028 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 2687
I20260809 16:29:41.156142 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:29:41.156306 30814 external_mini_cluster.cc:1428] Setting key 125736cda1a52ced2f420290baac9bcc
I20260809 16:29:41.158912 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:41.148048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:54693 (local address 127.30.23.190:34479)
0809 16:29:41.148114 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:41.148116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:41.148377 (+ 261us) server_negotiation.cc:407] Connection header received
0809 16:29:41.149014 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:41.149016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:41.149029 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:41.149102 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:41.150254 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:41.150900 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:41.151989 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:41.152189 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:41.152279 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:41.154832 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:41.154833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:41.154833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:41.154842 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:41.156689 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:41.157614 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:41.157616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:41.157616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:41.157680 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:41.158085 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:41.158086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:41.158086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:41.158238 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:41.158316 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:41.158721 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 16:29:41.158841 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:29:41.159189 2889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:41.148017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34479 (local address 127.30.23.130:54693)
0809 16:29:41.148208 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:41.148254 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:41.148871 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:41.149189 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:41.149193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:41.149921 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:41.150133 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:41.150138 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:41.151023 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:41.151024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:41.151868 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:41.151871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:41.152301 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:41.152303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:41.152439 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:41.153083 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:41.153093 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:41.154720 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:41.156844 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:41.156854 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:41.156856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:41.157243 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:41.157773 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:41.157775 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:41.157775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:41.157981 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:41.158321 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:41.158329 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:41.158527 (+ 198us) client_negotiation.cc:769] Sending connection context
0809 16:29:41.158654 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 16:29:41.158879 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":115,"threads_started":1}
I20260809 16:29:41.159829 2887 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34479
I20260809 16:29:41.159955 2887 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:41.160169 2887 heartbeater.cc:511] Master 127.30.23.190:34479 requested a full tablet report, sending...
I20260809 16:29:41.160687 2382 ts_manager.cc:194] Registered new tserver with Master: d0dc1da0ac6645628b409ed159152b8c (127.30.23.130:44781)
I20260809 16:29:41.161198 2382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:54693
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:29:41.190075 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34479
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:42081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:41.281867 2892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:41.282121 2892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:41.282164 2892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:41.283390 2892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:41.283437 2892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:41.283465 2892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:29:41.284935 2892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:42081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:41.285185 2892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:41.285487 2892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:41.290738 2901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:41.290848 2898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:41.290738 2899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:41.291041 2892 server_base.cc:1034] running on GCE node
I20260809 16:29:41.291307 2892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:41.291545 2892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:41.292773 2892 hybrid_clock.cc:648] HybridClock initialized: now 1786292981292760 us; error 31 us; skew 500 ppm
Aug 09 16:29:41 dist-test-slave-j2vl krb5kdc[2303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292981, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:41.295783 2892 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:29:41.296296 2892 webserver.cc:460] Webserver started at http://127.30.23.131:33069/ using document root <none> and password file <none>
I20260809 16:29:41.296540 2892 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:41.296586 2892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:41.296725 2892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:41.297883 2892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "75d8bab683cd45298274eb230493ca8d"
format_stamp: "Formatted at 2026-08-09 16:29:41 on dist-test-slave-j2vl"
server_key: "cf0780d1d9dcb5e876393d63a43c5bc8"
server_key_iv: "6e8c5a6c90c26ca08ea0a6631bc1fb78"
server_key_version: "encryptionkey@0"
I20260809 16:29:41.298249 2892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "75d8bab683cd45298274eb230493ca8d"
format_stamp: "Formatted at 2026-08-09 16:29:41 on dist-test-slave-j2vl"
server_key: "cf0780d1d9dcb5e876393d63a43c5bc8"
server_key_iv: "6e8c5a6c90c26ca08ea0a6631bc1fb78"
server_key_version: "encryptionkey@0"
I20260809 16:29:41.299686 2892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:29:41.300730 2914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:41.300987 2892 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:41.301105 2892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "75d8bab683cd45298274eb230493ca8d"
format_stamp: "Formatted at 2026-08-09 16:29:41 on dist-test-slave-j2vl"
server_key: "cf0780d1d9dcb5e876393d63a43c5bc8"
server_key_iv: "6e8c5a6c90c26ca08ea0a6631bc1fb78"
server_key_version: "encryptionkey@0"
I20260809 16:29:41.301184 2892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:41.315614 2892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:41.318109 2892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:41.318293 2892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:41.318555 2892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:41.318885 2892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:41.318941 2892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:41.318977 2892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:41.319007 2892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:41.325275 2892 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:41803
I20260809 16:29:41.325314 3095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:41803 every 8 connection(s)
I20260809 16:29:41.325661 2892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:29:41.326606 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 2892
I20260809 16:29:41.326754 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:29:41.326984 30814 external_mini_cluster.cc:1428] Setting key e52daafbf3f69fc25c1317498e1671e2
Aug 09 16:29:41 dist-test-slave-j2vl krb5kdc[2303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292981, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:41.336624 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:41.326450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:58037 (local address 127.30.23.190:34479)
0809 16:29:41.326508 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:41.326510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:41.326680 (+ 170us) server_negotiation.cc:407] Connection header received
0809 16:29:41.327365 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:41.327366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:41.327379 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:41.327434 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:41.328613 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:41.329157 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:41.330197 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:41.330405 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:41.330495 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:41.332891 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:41.332893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:41.332893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:41.332902 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:41.334877 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:41.335441 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:41.335442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:41.335443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:41.335515 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:41.335805 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:41.335806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:41.335807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:41.335984 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:41.336070 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:41.336423 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 16:29:41.336521 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:29:41.336872 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:41.326365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34479 (local address 127.30.23.131:58037)
0809 16:29:41.326528 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:41.326573 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:41.327242 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:41.327599 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:41.327604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:41.328308 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:41.328508 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:41.328513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:41.329267 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:41.329268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:41.330097 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:41.330099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:41.330506 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:41.330507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:41.330641 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:41.331237 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:41.331247 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:41.332776 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:41.335045 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:41.335056 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:41.335059 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:41.335325 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:41.335592 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:41.335593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:41.335593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:41.335719 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:41.336072 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:41.336079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:41.336235 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 16:29:41.336384 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 16:29:41.336573 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260809 16:29:41.337488 3096 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34479
I20260809 16:29:41.337615 3096 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:41.337778 3096 heartbeater.cc:511] Master 127.30.23.190:34479 requested a full tablet report, sending...
I20260809 16:29:41.338227 2382 ts_manager.cc:194] Registered new tserver with Master: 75d8bab683cd45298274eb230493ca8d (127.30.23.131:41803)
I20260809 16:29:41.338750 2382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:58037
I20260809 16:29:41.339350 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 16:29:41.354902 2442 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:29:41.353720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42636 (local address 127.30.23.190:34479)
0809 16:29:41.353836 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:41.353839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:41.353979 (+ 140us) server_negotiation.cc:407] Connection header received
0809 16:29:41.354015 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:41.354016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:41.354027 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:41.354091 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:41.354832 (+ 741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42636: BlockingRecv error: recv got EOF from 127.0.0.1:42636 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 16:29:41.355847 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 2478
I20260809 16:29:41.371456 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 2687
I20260809 16:29:41.386086 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 2892
I20260809 16:29:41.402207 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 2325
2026-08-09T16:29:41Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3623 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:29:41 dist-test-slave-j2vl krb5kdc[3119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:29:41 dist-test-slave-j2vl krb5kdc[3119](info): set up 2 sockets
Aug 09 16:29:41 dist-test-slave-j2vl krb5kdc[3119](info): commencing operation
krb5kdc: starting...
W20260809 16:29:42.709879 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:29:42 dist-test-slave-j2vl krb5kdc[3119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:29:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:29:42.850031 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:46799
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:40965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:46799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:42.933918 3142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:42.934166 3142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:42.934201 3142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:42.935355 3142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:42.935429 3142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:42.935456 3142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:42.935489 3142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:42.935518 3142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:42.937032 3142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:46799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:46799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:42.937306 3142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:42.937566 3142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:42.942709 3153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:42.942867 3142 server_base.cc:1034] running on GCE node
W20260809 16:29:42.942756 3149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:42.942721 3151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:42.943181 3142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:42.943499 3142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:42.944672 3142 hybrid_clock.cc:648] HybridClock initialized: now 1786292982944659 us; error 34 us; skew 500 ppm
Aug 09 16:29:42 dist-test-slave-j2vl krb5kdc[3119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292982, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:42.947276 3142 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:29:42.947690 3142 webserver.cc:460] Webserver started at http://127.30.23.190:38053/ using document root <none> and password file <none>
I20260809 16:29:42.947903 3142 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:42.947952 3142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:42.948047 3142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:42.949136 3142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "f3cd835ed8944ed1935d79c4a9f90a1c"
format_stamp: "Formatted at 2026-08-09 16:29:42 on dist-test-slave-j2vl"
server_key: "2e320affb30057d362300c35cc1a7e8d"
server_key_iv: "0ba141857a17282f0836ba459c655e5d"
server_key_version: "encryptionkey@0"
I20260809 16:29:42.949553 3142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "f3cd835ed8944ed1935d79c4a9f90a1c"
format_stamp: "Formatted at 2026-08-09 16:29:42 on dist-test-slave-j2vl"
server_key: "2e320affb30057d362300c35cc1a7e8d"
server_key_iv: "0ba141857a17282f0836ba459c655e5d"
server_key_version: "encryptionkey@0"
I20260809 16:29:42.950985 3142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:42.952059 3168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:42.952314 3142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:42.952411 3142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "f3cd835ed8944ed1935d79c4a9f90a1c"
format_stamp: "Formatted at 2026-08-09 16:29:42 on dist-test-slave-j2vl"
server_key: "2e320affb30057d362300c35cc1a7e8d"
server_key_iv: "0ba141857a17282f0836ba459c655e5d"
server_key_version: "encryptionkey@0"
I20260809 16:29:42.952497 3142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:42.966666 3142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:42.968986 3142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:42.969146 3142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:42.973169 3142 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:46799
I20260809 16:29:42.973228 3242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:46799 every 8 connection(s)
I20260809 16:29:42.973735 3142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:42.974484 3244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:42.975790 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 3142
I20260809 16:29:42.975924 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:42.976099 30814 external_mini_cluster.cc:1428] Setting key 041820d5992a7df9481a261fe63054a7
I20260809 16:29:42.977205 3244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c: Bootstrap starting.
I20260809 16:29:42.978046 3244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:42.978430 3244 log.cc:824] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:42.979246 3244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c: No bootstrap required, opened a new log
I20260809 16:29:42.981973 3244 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cd835ed8944ed1935d79c4a9f90a1c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 46799 } }
I20260809 16:29:42.982211 3244 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:42.982255 3244 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3cd835ed8944ed1935d79c4a9f90a1c, State: Initialized, Role: FOLLOWER
I20260809 16:29:42.982419 3244 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cd835ed8944ed1935d79c4a9f90a1c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 46799 } }
I20260809 16:29:42.982519 3244 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:42.982554 3244 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:42.982611 3244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:42.983381 3244 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cd835ed8944ed1935d79c4a9f90a1c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 46799 } }
I20260809 16:29:42.983471 3244 leader_election.cc:302] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3cd835ed8944ed1935d79c4a9f90a1c; no voters:
I20260809 16:29:42.983669 3244 leader_election.cc:288] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [CANDIDATE]: Term 1 election: Requested vote from peers
Aug 09 16:29:42 dist-test-slave-j2vl krb5kdc[3119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:42.983767 3250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:42.983942 3250 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [term 1 LEADER]: Becoming Leader. State: Replica: f3cd835ed8944ed1935d79c4a9f90a1c, State: Running, Role: LEADER
I20260809 16:29:42.983953 3244 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:42.984130 3250 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cd835ed8944ed1935d79c4a9f90a1c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 46799 } }
I20260809 16:29:42.984865 3252 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3cd835ed8944ed1935d79c4a9f90a1c. Latest consensus state: current_term: 1 leader_uuid: "f3cd835ed8944ed1935d79c4a9f90a1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cd835ed8944ed1935d79c4a9f90a1c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 46799 } } }
I20260809 16:29:42.985181 3251 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3cd835ed8944ed1935d79c4a9f90a1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cd835ed8944ed1935d79c4a9f90a1c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 46799 } } }
I20260809 16:29:42.985271 3251 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:42.985062 3252 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:42.985954 3270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:42.986457 3270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:42.988574 3247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:42.976699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46772 (local address 127.30.23.190:46799)
0809 16:29:42.976879 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:42.976886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:42.976911 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:29:42.977297 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:42.977303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:42.977417 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:42.977533 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:42.978422 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:42.979407 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:42.980495 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:42.980899 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:42.980992 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:42.984463 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:42.984468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:42.984468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:42.984480 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:42.986743 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:42.987243 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:42.987246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:42.987247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:42.987362 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:42.987584 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:42.987585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:42.987586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:42.987919 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:42.988103 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:42.988212 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 16:29:42.988352 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260809 16:29:42.989095 3270 catalog_manager.cc:1347] Generated new cluster ID: 03590dab61034b3bbc620bb55b3426f8
I20260809 16:29:42.989168 3270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:29:42.989781 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:29:42.996618 3270 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:42.997141 3270 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:43.001003 3270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3cd835ed8944ed1935d79c4a9f90a1c: Generated new TSK 0
I20260809 16:29:43.001194 3270 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:29:43.016613 3247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:43.012654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46780 (local address 127.30.23.190:46799)
0809 16:29:43.012713 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:43.012716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:43.012850 (+ 134us) server_negotiation.cc:407] Connection header received
0809 16:29:43.012923 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:43.012924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:43.012935 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:43.012989 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:43.013495 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:43.014034 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:43.015138 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:43.015343 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:43.015405 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:43.016140 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:43.016141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:43.016141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:43.016333 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:43.016350 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:43.016355 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:43.016417 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:43.016492 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 16:29:43.016515 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:29:43.019150 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 3142
2026-08-09T16:29:43Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1614 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:29:43 dist-test-slave-j2vl krb5kdc[3292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:29:43 dist-test-slave-j2vl krb5kdc[3292](info): set up 2 sockets
Aug 09 16:29:43 dist-test-slave-j2vl krb5kdc[3292](info): commencing operation
krb5kdc: starting...
W20260809 16:29:44.296285 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:29:44 dist-test-slave-j2vl krb5kdc[3292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:29:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:29:44.437963 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:42183
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:40069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:42183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:44.526554 3318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:44.526827 3318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:44.526868 3318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:44.528074 3318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:44.528148 3318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:44.528173 3318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:44.528196 3318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:44.528225 3318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:44.529851 3318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:42183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:42183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:44.530150 3318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:44.530405 3318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:44.535758 3326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:44.535770 3331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:44.535811 3318 server_base.cc:1034] running on GCE node
W20260809 16:29:44.535809 3329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:44.536273 3318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:44.536584 3318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:44.537756 3318 hybrid_clock.cc:648] HybridClock initialized: now 1786292984537720 us; error 50 us; skew 500 ppm
Aug 09 16:29:44 dist-test-slave-j2vl krb5kdc[3292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292984, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:44.543545 3318 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:29:44.544113 3318 webserver.cc:460] Webserver started at http://127.30.23.190:37047/ using document root <none> and password file <none>
I20260809 16:29:44.544379 3318 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:44.544449 3318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:44.544615 3318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:44.545827 3318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "e55941754c244f6b9d9b4815b0e45422"
format_stamp: "Formatted at 2026-08-09 16:29:44 on dist-test-slave-j2vl"
server_key: "948eaff8dfa77384f5a19c8aec24cf67"
server_key_iv: "13f5759febab1993b6c6410f5a963eca"
server_key_version: "encryptionkey@0"
I20260809 16:29:44.546206 3318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "e55941754c244f6b9d9b4815b0e45422"
format_stamp: "Formatted at 2026-08-09 16:29:44 on dist-test-slave-j2vl"
server_key: "948eaff8dfa77384f5a19c8aec24cf67"
server_key_iv: "13f5759febab1993b6c6410f5a963eca"
server_key_version: "encryptionkey@0"
I20260809 16:29:44.547654 3318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:29:44.548472 3340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:44.548695 3318 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:44.548808 3318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "e55941754c244f6b9d9b4815b0e45422"
format_stamp: "Formatted at 2026-08-09 16:29:44 on dist-test-slave-j2vl"
server_key: "948eaff8dfa77384f5a19c8aec24cf67"
server_key_iv: "13f5759febab1993b6c6410f5a963eca"
server_key_version: "encryptionkey@0"
I20260809 16:29:44.548885 3318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:44.559394 3318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:44.561925 3318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:44.562108 3318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:44.566752 3318 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:42183
I20260809 16:29:44.566831 3414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:42183 every 8 connection(s)
I20260809 16:29:44.567202 3318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:44.568374 3415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:44.570984 3415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422: Bootstrap starting.
I20260809 16:29:44.571655 3415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:44.572046 3415 log.cc:824] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:44.572876 3415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422: No bootstrap required, opened a new log
I20260809 16:29:44.574297 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 3318
I20260809 16:29:44.574415 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:44.574596 30814 external_mini_cluster.cc:1428] Setting key bea485d2f58d59aedf8bb6a0c60ee54d
I20260809 16:29:44.574909 3415 raft_consensus.cc:348] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55941754c244f6b9d9b4815b0e45422" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 42183 } }
I20260809 16:29:44.575094 3415 raft_consensus.cc:374] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:44.575161 3415 raft_consensus.cc:729] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e55941754c244f6b9d9b4815b0e45422, State: Initialized, Role: FOLLOWER
I20260809 16:29:44.575350 3415 consensus_queue.cc:260] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55941754c244f6b9d9b4815b0e45422" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 42183 } }
I20260809 16:29:44.575452 3415 raft_consensus.cc:388] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:44.575490 3415 raft_consensus.cc:482] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:44.575529 3415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:44.576337 3415 raft_consensus.cc:504] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55941754c244f6b9d9b4815b0e45422" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 42183 } }
I20260809 16:29:44.576464 3415 leader_election.cc:302] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e55941754c244f6b9d9b4815b0e45422; no voters:
I20260809 16:29:44.576619 3415 leader_election.cc:288] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:44.576696 3423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:44.576895 3423 raft_consensus.cc:686] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [term 1 LEADER]: Becoming Leader. State: Replica: e55941754c244f6b9d9b4815b0e45422, State: Running, Role: LEADER
I20260809 16:29:44.576894 3415 sys_catalog.cc:564] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:44.577075 3423 consensus_queue.cc:237] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55941754c244f6b9d9b4815b0e45422" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 42183 } }
I20260809 16:29:44.577500 3427 sys_catalog.cc:455] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e55941754c244f6b9d9b4815b0e45422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55941754c244f6b9d9b4815b0e45422" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 42183 } } }
I20260809 16:29:44.577517 3430 sys_catalog.cc:455] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e55941754c244f6b9d9b4815b0e45422. Latest consensus state: current_term: 1 leader_uuid: "e55941754c244f6b9d9b4815b0e45422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e55941754c244f6b9d9b4815b0e45422" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 42183 } } }
I20260809 16:29:44.577598 3427 sys_catalog.cc:458] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:44.577610 3430 sys_catalog.cc:458] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:44.577814 3445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:44.578400 3445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:44.580288 3445 catalog_manager.cc:1347] Generated new cluster ID: 5ff8fde9bcae4cd89ab4eb18c2d564f4
I20260809 16:29:44.580360 3445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:29:44 dist-test-slave-j2vl krb5kdc[3292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:44.586479 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:44.575201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60150 (local address 127.30.23.190:42183)
0809 16:29:44.575404 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:44.575413 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:44.575450 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:29:44.575580 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:44.575588 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:44.575648 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:44.575805 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:44.576411 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:44.577850 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:44.578915 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:44.579499 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:44.579634 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:44.582532 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:44.582536 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:44.582537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:44.582549 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:44.584627 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:44.585047 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:44.585050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:44.585051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:44.585179 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:44.585424 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:44.585425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:44.585425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:44.585841 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:44.586034 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:44.586112 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 16:29:44.586261 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260809 16:29:44.591073 3445 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:44.591806 3445 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:44.601171 3445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e55941754c244f6b9d9b4815b0e45422: Generated new TSK 0
I20260809 16:29:44.601447 3445 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:29:44.641893 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:42183
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:40069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:44.736192 3469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:44.736423 3469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:44.736449 3469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:44.737753 3469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:44.737800 3469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:44.737829 3469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:29:44.739734 3469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:42183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:44.740028 3469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:44.740338 3469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:44.746294 3479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:44.746295 3480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:44.746394 3469 server_base.cc:1034] running on GCE node
W20260809 16:29:44.746330 3482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:44.746814 3469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:44.747080 3469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:44.748299 3469 hybrid_clock.cc:648] HybridClock initialized: now 1786292984748251 us; error 66 us; skew 500 ppm
Aug 09 16:29:44 dist-test-slave-j2vl krb5kdc[3292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292984, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:44.751472 3469 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:29:44.752302 3469 webserver.cc:460] Webserver started at http://127.30.23.129:43897/ using document root <none> and password file <none>
I20260809 16:29:44.752599 3469 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:44.752697 3469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:44.752895 3469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:44.754184 3469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "bf5dd6df0f4742d19125d5f8e3eed8f2"
format_stamp: "Formatted at 2026-08-09 16:29:44 on dist-test-slave-j2vl"
server_key: "44423dda8747b3019d7525a0eef2ab2d"
server_key_iv: "e7e61059aa01766211e7af0c5a067206"
server_key_version: "encryptionkey@0"
I20260809 16:29:44.754616 3469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "bf5dd6df0f4742d19125d5f8e3eed8f2"
format_stamp: "Formatted at 2026-08-09 16:29:44 on dist-test-slave-j2vl"
server_key: "44423dda8747b3019d7525a0eef2ab2d"
server_key_iv: "e7e61059aa01766211e7af0c5a067206"
server_key_version: "encryptionkey@0"
I20260809 16:29:44.756235 3469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:29:44.757498 3498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:44.757746 3469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:44.757899 3469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "bf5dd6df0f4742d19125d5f8e3eed8f2"
format_stamp: "Formatted at 2026-08-09 16:29:44 on dist-test-slave-j2vl"
server_key: "44423dda8747b3019d7525a0eef2ab2d"
server_key_iv: "e7e61059aa01766211e7af0c5a067206"
server_key_version: "encryptionkey@0"
I20260809 16:29:44.757992 3469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:44.776100 3469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:44.779197 3469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:44.779381 3469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:44.779649 3469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:44.779997 3469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:44.780055 3469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:44.780093 3469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:44.780141 3469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:44.787137 3469 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:37121
I20260809 16:29:44.787178 3678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:37121 every 8 connection(s)
I20260809 16:29:44.787575 3469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:29:44.789263 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 3469
I20260809 16:29:44.789525 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:29:44.789750 30814 external_mini_cluster.cc:1428] Setting key 6e6817f0ad6d992bb75f0f8ac4d88107
Aug 09 16:29:44 dist-test-slave-j2vl krb5kdc[3292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292984, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:44.799618 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:44.788589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:42875 (local address 127.30.23.190:42183)
0809 16:29:44.788679 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:44.788681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:44.788856 (+ 175us) server_negotiation.cc:407] Connection header received
0809 16:29:44.789486 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:44.789488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:44.789508 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:44.789583 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:44.790854 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:44.791609 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:44.792807 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:44.793150 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:44.793236 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:44.796080 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:44.796082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:44.796083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:44.796093 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:44.797961 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:44.798446 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:44.798447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:44.798448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:44.798514 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:44.798811 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:44.798812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:44.798812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:44.798969 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:44.799056 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:44.799406 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 16:29:44.799511 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 16:29:44.799721 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:44.788522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:42183 (local address 127.30.23.129:42875)
0809 16:29:44.788708 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:44.788763 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:44.789258 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:44.789689 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:44.789693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:44.790493 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:44.790752 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:44.790757 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:44.791754 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:44.791755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:44.792711 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:44.792714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:44.793283 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:44.793284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:44.793577 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:44.794187 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:44.794197 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:44.795969 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:44.798069 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:44.798078 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:44.798080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:44.798347 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:44.798607 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:44.798608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:44.798608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:44.798728 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:44.799075 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:44.799083 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:44.799246 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 16:29:44.799348 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 16:29:44.799498 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
I20260809 16:29:44.800421 3679 heartbeater.cc:347] Connected to a master server at 127.30.23.190:42183
I20260809 16:29:44.800537 3679 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:44.800792 3679 heartbeater.cc:511] Master 127.30.23.190:42183 requested a full tablet report, sending...
I20260809 16:29:44.801676 3362 ts_manager.cc:194] Registered new tserver with Master: bf5dd6df0f4742d19125d5f8e3eed8f2 (127.30.23.129:37121)
I20260809 16:29:44.802338 3362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:42875
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:29:44.824632 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:42183
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:40069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:44.916112 3685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:44.916361 3685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:44.916399 3685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:44.917550 3685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:44.917623 3685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:44.917668 3685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:29:44.919138 3685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:42183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:44.919416 3685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:44.919656 3685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:44.924928 3691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:44.925040 3695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:44.925127 3692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:44.925184 3685 server_base.cc:1034] running on GCE node
I20260809 16:29:44.925915 3685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:44.926188 3685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:44.927376 3685 hybrid_clock.cc:648] HybridClock initialized: now 1786292984927352 us; error 34 us; skew 500 ppm
Aug 09 16:29:44 dist-test-slave-j2vl krb5kdc[3292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292984, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:44.930276 3685 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:29:44.930819 3685 webserver.cc:460] Webserver started at http://127.30.23.130:46275/ using document root <none> and password file <none>
I20260809 16:29:44.931066 3685 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:44.931136 3685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:44.931296 3685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:44.932511 3685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "cd6ed4a525624e469af5f783fe1eb7e8"
format_stamp: "Formatted at 2026-08-09 16:29:44 on dist-test-slave-j2vl"
server_key: "857c2665e9dfd1e9db06e90f5fd333ac"
server_key_iv: "240639c12707f5dd610a8fae419125f2"
server_key_version: "encryptionkey@0"
I20260809 16:29:44.932891 3685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "cd6ed4a525624e469af5f783fe1eb7e8"
format_stamp: "Formatted at 2026-08-09 16:29:44 on dist-test-slave-j2vl"
server_key: "857c2665e9dfd1e9db06e90f5fd333ac"
server_key_iv: "240639c12707f5dd610a8fae419125f2"
server_key_version: "encryptionkey@0"
I20260809 16:29:44.934441 3685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:29:44.935488 3702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:44.935786 3685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:44.935958 3685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "cd6ed4a525624e469af5f783fe1eb7e8"
format_stamp: "Formatted at 2026-08-09 16:29:44 on dist-test-slave-j2vl"
server_key: "857c2665e9dfd1e9db06e90f5fd333ac"
server_key_iv: "240639c12707f5dd610a8fae419125f2"
server_key_version: "encryptionkey@0"
I20260809 16:29:44.936071 3685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:44.948040 3685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:44.950760 3685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:44.950958 3685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:44.951228 3685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:44.951622 3685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:44.951681 3685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:44.951732 3685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:44.951769 3685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:44.958495 3685 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:45115
I20260809 16:29:44.958537 3880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:45115 every 8 connection(s)
I20260809 16:29:44.958959 3685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:29:44.961165 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 3685
I20260809 16:29:44.961297 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:29:44.961565 30814 external_mini_cluster.cc:1428] Setting key af560c4fc3f5fbc3f12cc32575f91986
Aug 09 16:29:44 dist-test-slave-j2vl krb5kdc[3292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292984, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:44.971078 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:44.959910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:53195 (local address 127.30.23.190:42183)
0809 16:29:44.960004 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:44.960007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:44.960263 (+ 256us) server_negotiation.cc:407] Connection header received
0809 16:29:44.960968 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:44.960970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:44.960985 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:44.961067 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:44.962459 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:44.963008 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:44.964181 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:44.964407 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:44.964500 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:44.967435 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:44.967437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:44.967438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:44.967448 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:44.969364 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:44.969901 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:44.969902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:44.969903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:44.969974 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:44.970326 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:44.970327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:44.970327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:44.970494 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:44.970574 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:44.970860 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 16:29:44.970966 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 16:29:44.971383 3886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:44.959833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:42183 (local address 127.30.23.130:53195)
0809 16:29:44.960084 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:44.960130 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:44.960812 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:44.961176 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:44.961181 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:44.962020 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:44.962354 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:44.962361 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:44.963126 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:44.963127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:44.964068 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:44.964070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:44.964509 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:44.964510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:44.964680 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:44.965323 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:44.965333 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:44.967317 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:44.969496 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:44.969505 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:44.969507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:44.969788 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:44.970119 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:44.970120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:44.970120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:44.970244 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:44.970574 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:44.970580 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:44.970718 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 16:29:44.970811 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 16:29:44.971059 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":106,"threads_started":1}
I20260809 16:29:44.972172 3881 heartbeater.cc:347] Connected to a master server at 127.30.23.190:42183
I20260809 16:29:44.972277 3881 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:44.972479 3881 heartbeater.cc:511] Master 127.30.23.190:42183 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
I20260809 16:29:44.972966 3362 ts_manager.cc:194] Registered new tserver with Master: cd6ed4a525624e469af5f783fe1eb7e8 (127.30.23.130:45115)
I20260809 16:29:44.973413 3362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:53195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:29:44.993438 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:42183
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:40069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:45.080504 3889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:45.080715 3889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:45.080739 3889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:45.081881 3889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:45.081928 3889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:45.081954 3889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:29:45.083309 3889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:42183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:45.083525 3889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:45.083714 3889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:45.088887 3896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:45.088915 3897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:45.089382 3902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:45.089445 3889 server_base.cc:1034] running on GCE node
I20260809 16:29:45.089665 3889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:45.089917 3889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:45.091096 3889 hybrid_clock.cc:648] HybridClock initialized: now 1786292985091088 us; error 36 us; skew 500 ppm
Aug 09 16:29:45 dist-test-slave-j2vl krb5kdc[3292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292985, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:45.094138 3889 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:29:45.094625 3889 webserver.cc:460] Webserver started at http://127.30.23.131:46637/ using document root <none> and password file <none>
I20260809 16:29:45.094887 3889 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:45.094959 3889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:45.095112 3889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:45.096216 3889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "434854b03b394e8f888a7778d52a5d5d"
format_stamp: "Formatted at 2026-08-09 16:29:45 on dist-test-slave-j2vl"
server_key: "8770cf2f8011b9dd037d7502ee069dd7"
server_key_iv: "67d9e0d0df70940c955c545c260bc0f4"
server_key_version: "encryptionkey@0"
I20260809 16:29:45.096738 3889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "434854b03b394e8f888a7778d52a5d5d"
format_stamp: "Formatted at 2026-08-09 16:29:45 on dist-test-slave-j2vl"
server_key: "8770cf2f8011b9dd037d7502ee069dd7"
server_key_iv: "67d9e0d0df70940c955c545c260bc0f4"
server_key_version: "encryptionkey@0"
I20260809 16:29:45.098351 3889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:29:45.099192 3920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:45.099465 3889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:45.099596 3889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "434854b03b394e8f888a7778d52a5d5d"
format_stamp: "Formatted at 2026-08-09 16:29:45 on dist-test-slave-j2vl"
server_key: "8770cf2f8011b9dd037d7502ee069dd7"
server_key_iv: "67d9e0d0df70940c955c545c260bc0f4"
server_key_version: "encryptionkey@0"
I20260809 16:29:45.099682 3889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:45.121963 3889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:45.124468 3889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:45.124660 3889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:45.124955 3889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:45.125296 3889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:45.125380 3889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:45.125447 3889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:45.125483 3889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:45.131757 3889 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:36707
I20260809 16:29:45.131796 4092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:36707 every 8 connection(s)
I20260809 16:29:45.132141 3889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:29:45 dist-test-slave-j2vl krb5kdc[3292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292985, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:45.139458 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 3889
I20260809 16:29:45.139570 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:29:45.139765 30814 external_mini_cluster.cc:1428] Setting key ad5ae505aa3b93f729575f28c42cb7fd
I20260809 16:29:45.142180 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:45.132919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:57495 (local address 127.30.23.190:42183)
0809 16:29:45.133004 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:45.133007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:45.133238 (+ 231us) server_negotiation.cc:407] Connection header received
0809 16:29:45.133854 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:45.133855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:45.133867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:45.133934 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:45.135026 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:45.135534 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:45.136527 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:45.136730 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:45.136791 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:45.139089 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:45.139091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:45.139091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:45.139099 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:45.140942 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:45.141302 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:45.141303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:45.141303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:45.141410 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:45.141640 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:45.141641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:45.141641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:45.141750 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:45.141824 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:45.142043 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 16:29:45.142115 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:29:45.142328 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:45.132891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:42183 (local address 127.30.23.131:57495)
0809 16:29:45.133082 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:45.133118 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:45.133724 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:45.134065 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:45.134069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:45.134685 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:45.134921 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:45.134926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:45.135632 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:45.135633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:45.136445 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:45.136447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:45.136826 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:45.136827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:45.136926 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:45.137486 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:45.137494 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:45.138979 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:45.141007 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:45.141012 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:45.141014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:45.141232 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:45.141486 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:45.141487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:45.141487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:45.141583 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:45.141808 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:45.141812 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:45.141929 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 16:29:45.142012 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 16:29:45.142131 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":112,"threads_started":1}
I20260809 16:29:45.142730 4095 heartbeater.cc:347] Connected to a master server at 127.30.23.190:42183
I20260809 16:29:45.142863 4095 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:45.143061 4095 heartbeater.cc:511] Master 127.30.23.190:42183 requested a full tablet report, sending...
I20260809 16:29:45.143467 3362 ts_manager.cc:194] Registered new tserver with Master: 434854b03b394e8f888a7778d52a5d5d (127.30.23.131:36707)
I20260809 16:29:45.143853 3362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:57495
I20260809 16:29:45.151851 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:29:45 dist-test-slave-j2vl krb5kdc[3292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292985, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 16:29:45 dist-test-slave-j2vl krb5kdc[3292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292985, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:45.180344 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:45.172261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60162 (local address 127.30.23.190:42183)
0809 16:29:45.172360 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:45.172362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:45.172458 (+ 96us) server_negotiation.cc:407] Connection header received
0809 16:29:45.172550 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:45.172551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:45.172563 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:45.172621 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:45.173287 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:45.173838 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:45.174540 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:45.174736 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:45.174793 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:45.177404 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:45.177406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:45.177407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:45.177418 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:45.179160 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:45.179555 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:45.179557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:45.179557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:45.179615 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:45.179843 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:45.179844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:45.179845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:45.179998 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:45.180077 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:45.180182 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 16:29:45.180246 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 16:29:45.180649 3362 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60162
I20260809 16:29:45.181119 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 3469
I20260809 16:29:45.200067 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 3685
I20260809 16:29:45.213783 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 3889
I20260809 16:29:45.222884 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 3318
2026-08-09T16:29:45Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2206 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:29:45 dist-test-slave-j2vl krb5kdc[4110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:29:45 dist-test-slave-j2vl krb5kdc[4110](info): set up 2 sockets
Aug 09 16:29:45 dist-test-slave-j2vl krb5kdc[4110](info): commencing operation
krb5kdc: starting...
W20260809 16:29:46.533538 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:29:46 dist-test-slave-j2vl krb5kdc[4110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:29:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:29:46.669695 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:45059
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:45059
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:46.752900 4138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:46.753125 4138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:46.753150 4138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:46.754298 4138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:46.754338 4138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:46.754351 4138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:46.754364 4138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:46.754376 4138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:46.755858 4138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:45059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:45059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:46.756060 4138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:46.756234 4138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:46.761390 4149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:46.761379 4154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:46.761482 4138 server_base.cc:1034] running on GCE node
W20260809 16:29:46.761375 4150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:46.761883 4138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:46.762158 4138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:46.763310 4138 hybrid_clock.cc:648] HybridClock initialized: now 1786292986763287 us; error 44 us; skew 500 ppm
Aug 09 16:29:46 dist-test-slave-j2vl krb5kdc[4110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292986, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:46.766101 4138 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:29:46.766537 4138 webserver.cc:460] Webserver started at http://127.30.23.190:37749/ using document root <none> and password file <none>
I20260809 16:29:46.766773 4138 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:46.766844 4138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:46.767000 4138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:46.768112 4138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "9b9c944fafec4eb8911b85c1b6e882a7"
format_stamp: "Formatted at 2026-08-09 16:29:46 on dist-test-slave-j2vl"
server_key: "79301f1a7552d7ced89fc53c61219ab9"
server_key_iv: "6959d94bcebedcfd08771c7bbdf163d9"
server_key_version: "encryptionkey@0"
I20260809 16:29:46.768494 4138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "9b9c944fafec4eb8911b85c1b6e882a7"
format_stamp: "Formatted at 2026-08-09 16:29:46 on dist-test-slave-j2vl"
server_key: "79301f1a7552d7ced89fc53c61219ab9"
server_key_iv: "6959d94bcebedcfd08771c7bbdf163d9"
server_key_version: "encryptionkey@0"
I20260809 16:29:46.769987 4138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:29:46.770890 4164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:46.771139 4138 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:46.771277 4138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "9b9c944fafec4eb8911b85c1b6e882a7"
format_stamp: "Formatted at 2026-08-09 16:29:46 on dist-test-slave-j2vl"
server_key: "79301f1a7552d7ced89fc53c61219ab9"
server_key_iv: "6959d94bcebedcfd08771c7bbdf163d9"
server_key_version: "encryptionkey@0"
I20260809 16:29:46.771356 4138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:46.787225 4138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:46.789770 4138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:46.789956 4138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:46.794476 4138 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:45059
I20260809 16:29:46.794517 4247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:45059 every 8 connection(s)
I20260809 16:29:46.795085 4138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:46.795476 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 4138
I20260809 16:29:46.795595 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:46.795775 30814 external_mini_cluster.cc:1428] Setting key 531a35305f78fde4f2b5ef164b0bb093
I20260809 16:29:46.796175 4248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:46.800102 4248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7: Bootstrap starting.
I20260809 16:29:46.800859 4248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:46.801198 4248 log.cc:824] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:46.802031 4248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7: No bootstrap required, opened a new log
Aug 09 16:29:46 dist-test-slave-j2vl krb5kdc[4110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:46.804581 4248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9c944fafec4eb8911b85c1b6e882a7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45059 } }
I20260809 16:29:46.804731 4248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:46.804811 4248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b9c944fafec4eb8911b85c1b6e882a7, State: Initialized, Role: FOLLOWER
I20260809 16:29:46.804970 4248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9c944fafec4eb8911b85c1b6e882a7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45059 } }
I20260809 16:29:46.805069 4248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:46.805115 4248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:46.805181 4248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:46.806054 4248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9c944fafec4eb8911b85c1b6e882a7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45059 } }
I20260809 16:29:46.806188 4248 leader_election.cc:302] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b9c944fafec4eb8911b85c1b6e882a7; no voters:
I20260809 16:29:46.806421 4248 leader_election.cc:288] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:46.806456 4256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:46.806572 4256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [term 1 LEADER]: Becoming Leader. State: Replica: 9b9c944fafec4eb8911b85c1b6e882a7, State: Running, Role: LEADER
I20260809 16:29:46.806725 4256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9c944fafec4eb8911b85c1b6e882a7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45059 } }
I20260809 16:29:46.806728 4248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:46.807154 4259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b9c944fafec4eb8911b85c1b6e882a7. Latest consensus state: current_term: 1 leader_uuid: "9b9c944fafec4eb8911b85c1b6e882a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9c944fafec4eb8911b85c1b6e882a7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45059 } } }
I20260809 16:29:46.807159 4257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b9c944fafec4eb8911b85c1b6e882a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9c944fafec4eb8911b85c1b6e882a7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45059 } } }
I20260809 16:29:46.807248 4259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:46.807252 4257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:46.807490 4262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:46.808089 4262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:46.808471 4253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:46.796387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49242 (local address 127.30.23.190:45059)
0809 16:29:46.796598 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:46.796609 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:46.796648 (+ 39us) server_negotiation.cc:407] Connection header received
0809 16:29:46.797594 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:46.797605 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:46.797676 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:46.797848 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:46.798510 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:46.800043 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:46.800877 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:46.801576 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:46.801668 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:46.804521 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:46.804525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:46.804525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:46.804536 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:46.806546 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:46.806966 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:46.806969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:46.806970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:46.807084 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:46.807306 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:46.807307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:46.807307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:46.807679 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:46.807835 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:46.808060 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 16:29:46.808234 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":91,"threads_started":1}
I20260809 16:29:46.810168 4262 catalog_manager.cc:1347] Generated new cluster ID: c8dad8a9a3a14c5b9d279644283768fc
I20260809 16:29:46.810231 4262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:29:46.845713 4262 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:46.846575 4262 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:46.855376 4262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b9c944fafec4eb8911b85c1b6e882a7: Generated new TSK 0
I20260809 16:29:46.855597 4262 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:29:46.885372 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45059
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:46.969470 4297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:46.969717 4297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:46.969759 4297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:46.971385 4297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:46.971478 4297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:46.971534 4297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:29:46.973369 4297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:46.973687 4297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:46.973964 4297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:46.979513 4307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:46.979488 4311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:46.979750 4297 server_base.cc:1034] running on GCE node
W20260809 16:29:46.979492 4305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:46.980018 4297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:46.980293 4297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:46.981431 4297 hybrid_clock.cc:648] HybridClock initialized: now 1786292986981385 us; error 56 us; skew 500 ppm
Aug 09 16:29:46 dist-test-slave-j2vl krb5kdc[4110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292986, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:46.984503 4297 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:29:46.984972 4297 webserver.cc:460] Webserver started at http://127.30.23.129:39571/ using document root <none> and password file <none>
I20260809 16:29:46.985229 4297 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:46.985302 4297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:46.985523 4297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:46.986637 4297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "2fa19c79c58046888ea047f98cd0e67d"
format_stamp: "Formatted at 2026-08-09 16:29:46 on dist-test-slave-j2vl"
server_key: "4933ff97dee2dad90bb20212513b9c8e"
server_key_iv: "8cb6b0e01b317114a998de38df356c96"
server_key_version: "encryptionkey@0"
I20260809 16:29:46.987011 4297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "2fa19c79c58046888ea047f98cd0e67d"
format_stamp: "Formatted at 2026-08-09 16:29:46 on dist-test-slave-j2vl"
server_key: "4933ff97dee2dad90bb20212513b9c8e"
server_key_iv: "8cb6b0e01b317114a998de38df356c96"
server_key_version: "encryptionkey@0"
I20260809 16:29:46.988520 4297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:29:46.989320 4323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:46.989552 4297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:46.989660 4297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "2fa19c79c58046888ea047f98cd0e67d"
format_stamp: "Formatted at 2026-08-09 16:29:46 on dist-test-slave-j2vl"
server_key: "4933ff97dee2dad90bb20212513b9c8e"
server_key_iv: "8cb6b0e01b317114a998de38df356c96"
server_key_version: "encryptionkey@0"
I20260809 16:29:46.989741 4297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:47.011107 4297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:47.013209 4297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:47.013459 4297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:47.013717 4297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:47.014050 4297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:47.014108 4297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:47.014150 4297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:47.014180 4297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:47.020164 4297 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:43475
I20260809 16:29:47.020217 4503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:43475 every 8 connection(s)
I20260809 16:29:47.020550 4297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:29:47.021054 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 4297
I20260809 16:29:47.021168 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:29:47.021436 30814 external_mini_cluster.cc:1428] Setting key 6319d5bdf4c8f0f3219828387b11b6a4
Aug 09 16:29:47 dist-test-slave-j2vl krb5kdc[4110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292986, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
I20260809 16:29:47.031709 4253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:47.021326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:52301 (local address 127.30.23.190:45059)
0809 16:29:47.021393 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:47.021396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:47.021597 (+ 201us) server_negotiation.cc:407] Connection header received
0809 16:29:47.022898 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:47.022899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:47.022911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:47.022978 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:47.023889 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.024732 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.025899 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.026123 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.026214 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:47.028562 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:47.028564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:47.028564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:47.028577 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:47.030141 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:47.030549 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:47.030550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:47.030551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:47.030611 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:47.030991 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:47.030992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:47.030993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:47.031127 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:47.031211 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:47.031496 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 16:29:47.031606 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:29:47.031955 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:47.021289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:45059 (local address 127.30.23.129:52301)
0809 16:29:47.021468 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:47.021511 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:47.022745 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:47.023062 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:47.023066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:47.023486 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:47.023775 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:47.023781 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.024843 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.024844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:47.025799 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:47.025802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.026233 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.026234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:47.026364 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:47.026981 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:47.026991 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:47.028450 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:47.030229 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:47.030237 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:47.030240 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:47.030470 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:47.030713 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:47.030714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:47.030714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:47.030892 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:47.031202 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:47.031209 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:47.031352 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 16:29:47.031462 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 16:29:47.031670 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260809 16:29:47.032590 4504 heartbeater.cc:347] Connected to a master server at 127.30.23.190:45059
I20260809 16:29:47.032732 4504 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:47.032968 4504 heartbeater.cc:511] Master 127.30.23.190:45059 requested a full tablet report, sending...
I20260809 16:29:47.033591 4193 ts_manager.cc:194] Registered new tserver with Master: 2fa19c79c58046888ea047f98cd0e67d (127.30.23.129:43475)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
I20260809 16:29:47.034411 4193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:52301
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:29:47.052927 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45059
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:47.139328 4513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:47.139526 4513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:47.139549 4513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:47.140631 4513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:47.140668 4513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:47.140694 4513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:29:47.142194 4513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:47.142383 4513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:47.142556 4513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:47.147645 4530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:47.147646 4527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:47.147675 4533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:47.147861 4513 server_base.cc:1034] running on GCE node
I20260809 16:29:47.148159 4513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:47.148416 4513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:47.149601 4513 hybrid_clock.cc:648] HybridClock initialized: now 1786292987149578 us; error 39 us; skew 500 ppm
Aug 09 16:29:47 dist-test-slave-j2vl krb5kdc[4110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292987, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:47.152122 4513 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:29:47.152518 4513 webserver.cc:460] Webserver started at http://127.30.23.130:37865/ using document root <none> and password file <none>
I20260809 16:29:47.152699 4513 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:47.152740 4513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:47.152830 4513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:47.153901 4513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "70ade5cda2884c61b42a9df29c189c2d"
format_stamp: "Formatted at 2026-08-09 16:29:47 on dist-test-slave-j2vl"
server_key: "7feb7bcb4c23a0570065e774fc194d06"
server_key_iv: "a7f33b9308e97a0d7ee559b6a6724be0"
server_key_version: "encryptionkey@0"
I20260809 16:29:47.154208 4513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "70ade5cda2884c61b42a9df29c189c2d"
format_stamp: "Formatted at 2026-08-09 16:29:47 on dist-test-slave-j2vl"
server_key: "7feb7bcb4c23a0570065e774fc194d06"
server_key_iv: "a7f33b9308e97a0d7ee559b6a6724be0"
server_key_version: "encryptionkey@0"
I20260809 16:29:47.155507 4513 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:29:47.156253 4545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:47.156502 4513 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:47.156612 4513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "70ade5cda2884c61b42a9df29c189c2d"
format_stamp: "Formatted at 2026-08-09 16:29:47 on dist-test-slave-j2vl"
server_key: "7feb7bcb4c23a0570065e774fc194d06"
server_key_iv: "a7f33b9308e97a0d7ee559b6a6724be0"
server_key_version: "encryptionkey@0"
I20260809 16:29:47.156687 4513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:47.175000 4513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:47.177568 4513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:47.177732 4513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:47.177996 4513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:47.178335 4513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:47.178372 4513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:47.178395 4513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:47.178408 4513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:47.184373 4513 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:34299
I20260809 16:29:47.184453 4730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:34299 every 8 connection(s)
I20260809 16:29:47.184788 4513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:29:47.189090 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 4513
I20260809 16:29:47.189182 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:29:47.189447 30814 external_mini_cluster.cc:1428] Setting key 55c151e166098a7d2a4fcd5ed633672c
Aug 09 16:29:47 dist-test-slave-j2vl krb5kdc[4110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292987, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:47.194662 4253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:47.185612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:58381 (local address 127.30.23.190:45059)
0809 16:29:47.185675 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:47.185677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:47.185903 (+ 226us) server_negotiation.cc:407] Connection header received
0809 16:29:47.186894 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:47.186896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:47.186909 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:47.186964 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:47.187581 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.188101 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.189101 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.189443 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.189530 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:47.191681 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:47.191682 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:47.191683 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:47.191691 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:47.193170 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:47.193706 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:47.193708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:47.193708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:47.193765 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:47.194054 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:47.194055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:47.194055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:47.194157 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:47.194209 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:47.194529 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 16:29:47.194598 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:29:47.195080 4740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:47.185594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:45059 (local address 127.30.23.130:58381)
0809 16:29:47.185777 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:47.185814 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:47.186780 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:47.187033 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:47.187042 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:47.187300 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:47.187483 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:47.187487 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.188203 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.188204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:47.189005 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:47.189007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.189542 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.189544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:47.189706 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:47.190237 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:47.190258 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:47.191584 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:47.193265 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:47.193274 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:47.193276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:47.193634 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:47.193841 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:47.193842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:47.193842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:47.193990 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:47.194220 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:47.194226 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:47.194395 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 16:29:47.194512 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 16:29:47.194727 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
I20260809 16:29:47.195613 4734 heartbeater.cc:347] Connected to a master server at 127.30.23.190:45059
I20260809 16:29:47.195719 4734 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:47.195894 4734 heartbeater.cc:511] Master 127.30.23.190:45059 requested a full tablet report, sending...
I20260809 16:29:47.196319 4193 ts_manager.cc:194] Registered new tserver with Master: 70ade5cda2884c61b42a9df29c189c2d (127.30.23.130:34299)
I20260809 16:29:47.196739 4193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:58381
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:29:47.220419 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45059
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:47.309980 4744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:47.310195 4744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:47.310220 4744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:47.311334 4744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:47.311370 4744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:47.311396 4744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:29:47.312853 4744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:47.313059 4744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:47.313237 4744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:47.318719 4753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:47.318706 4754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:47.318703 4758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:47.318737 4744 server_base.cc:1034] running on GCE node
I20260809 16:29:47.319169 4744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:47.319428 4744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:47.320614 4744 hybrid_clock.cc:648] HybridClock initialized: now 1786292987320585 us; error 50 us; skew 500 ppm
Aug 09 16:29:47 dist-test-slave-j2vl krb5kdc[4110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292987, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:47.323235 4744 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:29:47.323716 4744 webserver.cc:460] Webserver started at http://127.30.23.131:44617/ using document root <none> and password file <none>
I20260809 16:29:47.323927 4744 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:47.323968 4744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:47.324059 4744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:47.325047 4744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "a538a9f7dc7344f89a37896e4c26463e"
format_stamp: "Formatted at 2026-08-09 16:29:47 on dist-test-slave-j2vl"
server_key: "0a38ab11618e68ca7483a0784553421f"
server_key_iv: "7b092f36ad729e79fe5a5ebce9be3cd1"
server_key_version: "encryptionkey@0"
I20260809 16:29:47.325402 4744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "a538a9f7dc7344f89a37896e4c26463e"
format_stamp: "Formatted at 2026-08-09 16:29:47 on dist-test-slave-j2vl"
server_key: "0a38ab11618e68ca7483a0784553421f"
server_key_iv: "7b092f36ad729e79fe5a5ebce9be3cd1"
server_key_version: "encryptionkey@0"
I20260809 16:29:47.326835 4744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:47.327621 4770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:47.327785 4744 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:29:47.327872 4744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "a538a9f7dc7344f89a37896e4c26463e"
format_stamp: "Formatted at 2026-08-09 16:29:47 on dist-test-slave-j2vl"
server_key: "0a38ab11618e68ca7483a0784553421f"
server_key_iv: "7b092f36ad729e79fe5a5ebce9be3cd1"
server_key_version: "encryptionkey@0"
I20260809 16:29:47.327950 4744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:47.339241 4744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:47.341281 4744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:47.341480 4744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:47.341692 4744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:47.342020 4744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:47.342053 4744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:47.342077 4744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:47.342124 4744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:47.348088 4744 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:36475
I20260809 16:29:47.348172 4944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:36475 every 8 connection(s)
I20260809 16:29:47.348502 4744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:29:47 dist-test-slave-j2vl krb5kdc[4110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292987, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:47.357126 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 4744
I20260809 16:29:47.357246 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:29:47.357487 30814 external_mini_cluster.cc:1428] Setting key 2012813b4ba442e05ea98a526f796835
I20260809 16:29:47.359107 4253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:47.349326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:44303 (local address 127.30.23.190:45059)
0809 16:29:47.349454 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:47.349462 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:47.349637 (+ 175us) server_negotiation.cc:407] Connection header received
0809 16:29:47.350600 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:47.350602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:47.350612 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:47.350677 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:47.351292 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.351789 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.352841 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.353056 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.353116 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:47.355666 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:47.355668 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:47.355669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:47.355676 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:47.357335 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:47.357988 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:47.357989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:47.357989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:47.358068 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:47.358417 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:47.358418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:47.358418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:47.358578 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:47.358667 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:47.358964 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 16:29:47.359041 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 16:29:47.359233 4950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:47.349235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:45059 (local address 127.30.23.131:44303)
0809 16:29:47.349500 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:47.349544 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:47.350486 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:47.350751 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:47.350754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:47.351021 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:47.351214 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:47.351217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.351893 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.351894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:47.352732 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:47.352734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.353142 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.353143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:47.353275 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:47.353909 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:47.353923 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:47.355553 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:47.357484 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:47.357491 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:47.357493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:47.357853 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:47.358156 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:47.358156 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:47.358157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:47.358342 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:47.358651 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:47.358658 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:47.358812 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 16:29:47.358904 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 16:29:47.359038 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":192,"threads_started":1}
I20260809 16:29:47.359683 4945 heartbeater.cc:347] Connected to a master server at 127.30.23.190:45059
I20260809 16:29:47.359802 4945 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:47.359989 4945 heartbeater.cc:511] Master 127.30.23.190:45059 requested a full tablet report, sending...
I20260809 16:29:47.360467 4193 ts_manager.cc:194] Registered new tserver with Master: a538a9f7dc7344f89a37896e4c26463e (127.30.23.131:36475)
I20260809 16:29:47.360877 4193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:44303
I20260809 16:29:47.369951 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:29:47 dist-test-slave-j2vl krb5kdc[4110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:29:47 dist-test-slave-j2vl krb5kdc[4110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.129@KRBTEST.COM
I20260809 16:29:47.395674 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:47.387108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36444 (local address 127.30.23.129:43475)
0809 16:29:47.387276 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:47.387281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:47.387292 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:29:47.387320 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:47.387322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:47.387336 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:47.387414 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:47.388050 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.388761 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.389617 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.389988 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.390055 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:47.392566 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:47.392577 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:47.392578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:47.392584 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:47.394441 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:47.394878 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:47.394880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:47.394881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:47.394937 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:47.395178 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:47.395179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:47.395179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:47.395335 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:47.395454 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:47.395524 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 16:29:47.395592 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
Aug 09 16:29:47 dist-test-slave-j2vl krb5kdc[4110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:47.404405 4253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:47.397038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49248 (local address 127.30.23.190:45059)
0809 16:29:47.397118 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:47.397120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:47.397133 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:29:47.397203 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:47.397205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:47.397215 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:47.397273 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:47.397771 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.398265 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.399018 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:47.399224 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:47.399310 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:47.401697 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:47.401699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:47.401700 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:47.401710 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:47.403094 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:47.403466 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:47.403468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:47.403468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:47.403538 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:47.403769 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:47.403770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:47.403770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:47.403948 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:47.404068 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:47.404186 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 16:29:47.404282 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 16:29:47.404726 4193 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49248
I20260809 16:29:47.405133 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 4297
I20260809 16:29:47.416296 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 4513
I20260809 16:29:47.427567 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 4744
I20260809 16:29:47.450682 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 4138
2026-08-09T16:29:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2228 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:29:47 dist-test-slave-j2vl krb5kdc[4967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:29:47 dist-test-slave-j2vl krb5kdc[4967](info): set up 2 sockets
Aug 09 16:29:47 dist-test-slave-j2vl krb5kdc[4967](info): commencing operation
krb5kdc: starting...
W20260809 16:29:48.735509 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:29:48 dist-test-slave-j2vl krb5kdc[4967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:29:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:29:48.864914 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:33281
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:33281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:48.950781 4988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:48.951043 4988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:48.951083 4988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:48.952280 4988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:48.952358 4988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:48.952383 4988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:48.952406 4988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:48.952435 4988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:48.954003 4988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:33281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:33281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:48.954299 4988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:48.954548 4988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:48.959826 5003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:48.959792 5002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:48.959798 5006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:48.959836 4988 server_base.cc:1034] running on GCE node
I20260809 16:29:48.960285 4988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:48.960592 4988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:48.961738 4988 hybrid_clock.cc:648] HybridClock initialized: now 1786292988961725 us; error 33 us; skew 500 ppm
Aug 09 16:29:48 dist-test-slave-j2vl krb5kdc[4967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292988, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:48.964340 4988 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:29:48.964736 4988 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:48.964809 4988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:48.964982 4988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:48.966173 4988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "f1b88da8686d44c691106d241db34819"
format_stamp: "Formatted at 2026-08-09 16:29:48 on dist-test-slave-j2vl"
server_key: "cd25e488aef8f5d32279a1ce01e44759"
server_key_iv: "19becfd1918a4bed7f634c4105791527"
server_key_version: "encryptionkey@0"
I20260809 16:29:48.966553 4988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "f1b88da8686d44c691106d241db34819"
format_stamp: "Formatted at 2026-08-09 16:29:48 on dist-test-slave-j2vl"
server_key: "cd25e488aef8f5d32279a1ce01e44759"
server_key_iv: "19becfd1918a4bed7f634c4105791527"
server_key_version: "encryptionkey@0"
I20260809 16:29:48.967936 4988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:29:48.968813 5014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:48.969048 4988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:48.969165 4988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "f1b88da8686d44c691106d241db34819"
format_stamp: "Formatted at 2026-08-09 16:29:48 on dist-test-slave-j2vl"
server_key: "cd25e488aef8f5d32279a1ce01e44759"
server_key_iv: "19becfd1918a4bed7f634c4105791527"
server_key_version: "encryptionkey@0"
I20260809 16:29:48.969252 4988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:48.993244 4988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:48.995630 4988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:48.995826 4988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:49.000362 4988 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:33281
I20260809 16:29:49.000445 5103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:33281 every 8 connection(s)
I20260809 16:29:49.000756 4988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:49.001386 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 4988
I20260809 16:29:49.001555 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:49.001813 30814 external_mini_cluster.cc:1428] Setting key e70fcea284d2dff908538be42bce6d73
I20260809 16:29:49.002135 5104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:49.005126 5104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819: Bootstrap starting.
I20260809 16:29:49.005846 5104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:49.006155 5104 log.cc:824] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:49.007014 5104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819: No bootstrap required, opened a new log
Aug 09 16:29:49 dist-test-slave-j2vl krb5kdc[4967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:49.009663 5104 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b88da8686d44c691106d241db34819" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33281 } }
I20260809 16:29:49.009836 5104 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:49.009907 5104 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1b88da8686d44c691106d241db34819, State: Initialized, Role: FOLLOWER
I20260809 16:29:49.010054 5104 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b88da8686d44c691106d241db34819" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33281 } }
I20260809 16:29:49.010149 5104 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:49.010185 5104 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:49.010243 5104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:49.010996 5104 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b88da8686d44c691106d241db34819" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33281 } }
I20260809 16:29:49.011114 5104 leader_election.cc:302] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1b88da8686d44c691106d241db34819; no voters:
I20260809 16:29:49.011299 5104 leader_election.cc:288] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:49.011353 5111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:49.011554 5111 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [term 1 LEADER]: Becoming Leader. State: Replica: f1b88da8686d44c691106d241db34819, State: Running, Role: LEADER
I20260809 16:29:49.011583 5104 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:49.011650 5111 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b88da8686d44c691106d241db34819" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33281 } }
I20260809 16:29:49.012018 5111 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1b88da8686d44c691106d241db34819. Latest consensus state: current_term: 1 leader_uuid: "f1b88da8686d44c691106d241db34819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b88da8686d44c691106d241db34819" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33281 } } }
I20260809 16:29:49.012256 5111 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:49.012181 5114 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1b88da8686d44c691106d241db34819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b88da8686d44c691106d241db34819" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33281 } } }
I20260809 16:29:49.012691 5114 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819 [sys.catalog]: This master's current role is: LEADER
W20260809 16:29:49.013530 5141 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 16:29:49.013545 5107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.002348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60698 (local address 127.30.23.190:33281)
0809 16:29:49.002560 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:49.002568 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:49.002594 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:29:49.003132 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:49.003139 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:49.003188 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:49.003342 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:49.004196 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.005090 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.005939 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.006443 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.006564 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.009273 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:49.009278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:49.009279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:49.009294 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:49.011424 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:49.011922 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:49.011924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:49.011925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:49.012043 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:49.012300 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:49.012301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:49.012302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:49.012631 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:49.012863 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:49.012993 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 16:29:49.013268 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
W20260809 16:29:49.013607 5141 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:29:49.013675 5128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:49.013819 5128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:49.015368 5128 catalog_manager.cc:1347] Generated new cluster ID: 65c9868ff2bd45b792f411f73aae122a
I20260809 16:29:49.015426 5128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:29:49.020594 5128 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:49.021317 5128 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:49.031304 5128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819: Generated new TSK 0
I20260809 16:29:49.031528 5128 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:29:49.073889 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33281
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:49.178239 5149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:49.178457 5149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:49.178483 5149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:49.179610 5149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:49.179648 5149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:49.179674 5149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:29:49.181257 5149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:49.181519 5149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:49.181715 5149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:49.187067 5160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:49.187067 5158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:49.187093 5157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:49.187314 5149 server_base.cc:1034] running on GCE node
I20260809 16:29:49.187721 5149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:49.187990 5149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:49.189208 5149 hybrid_clock.cc:648] HybridClock initialized: now 1786292989189202 us; error 39 us; skew 500 ppm
Aug 09 16:29:49 dist-test-slave-j2vl krb5kdc[4967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292989, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:49.192123 5149 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:29:49.192541 5149 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:49.192617 5149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:49.192772 5149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:49.193961 5149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "fa2eaeb699c7412b99455b5ca0c494cc"
format_stamp: "Formatted at 2026-08-09 16:29:49 on dist-test-slave-j2vl"
server_key: "a99d21d1b1d34ddf08357a87a932a7a1"
server_key_iv: "272dedf0c9c6960cf141d80ddb01ba26"
server_key_version: "encryptionkey@0"
I20260809 16:29:49.194337 5149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "fa2eaeb699c7412b99455b5ca0c494cc"
format_stamp: "Formatted at 2026-08-09 16:29:49 on dist-test-slave-j2vl"
server_key: "a99d21d1b1d34ddf08357a87a932a7a1"
server_key_iv: "272dedf0c9c6960cf141d80ddb01ba26"
server_key_version: "encryptionkey@0"
I20260809 16:29:49.195688 5149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:29:49.196588 5177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:49.196811 5149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:49.196935 5149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "fa2eaeb699c7412b99455b5ca0c494cc"
format_stamp: "Formatted at 2026-08-09 16:29:49 on dist-test-slave-j2vl"
server_key: "a99d21d1b1d34ddf08357a87a932a7a1"
server_key_iv: "272dedf0c9c6960cf141d80ddb01ba26"
server_key_version: "encryptionkey@0"
I20260809 16:29:49.197017 5149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:49.214325 5149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:49.216873 5149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:49.217051 5149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:49.217304 5149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:49.217680 5149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:49.217751 5149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:49.217799 5149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:49.217831 5149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:49.223937 5149 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:41043
I20260809 16:29:49.224035 5368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:41043 every 8 connection(s)
I20260809 16:29:49.224324 5149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:29:49.231073 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 5149
Aug 09 16:29:49 dist-test-slave-j2vl krb5kdc[4967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292989, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:49.231220 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:29:49.231436 30814 external_mini_cluster.cc:1428] Setting key 83b70bfb9bf967f5221f50ad83188d8b
I20260809 16:29:49.234822 5107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.225146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:54769 (local address 127.30.23.190:33281)
0809 16:29:49.225198 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:49.225201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:49.225428 (+ 227us) server_negotiation.cc:407] Connection header received
0809 16:29:49.226123 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:49.226124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:49.226139 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:49.226199 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:49.227201 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.227723 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.228802 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.229034 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.229114 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.231667 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:49.231668 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:49.231669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:49.231678 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:49.233186 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:49.233736 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:49.233737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:49.233738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:49.233797 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:49.234121 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:49.234126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:49.234127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:49.234240 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:49.234324 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:49.234616 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 16:29:49.234730 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:29:49.235056 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.225101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33281 (local address 127.30.23.129:54769)
0809 16:29:49.225257 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:49.225308 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:49.225983 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:49.226267 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:49.226272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:49.226568 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:49.227077 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.227082 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.227845 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.227846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.228695 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.228698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.229126 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.229127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.229264 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.229859 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:49.229868 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:49.231557 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:49.233275 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:49.233284 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:49.233287 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:49.233653 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:49.233902 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:49.233903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:49.233903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:49.234036 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:49.234308 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:49.234316 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:49.234473 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 16:29:49.234582 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 16:29:49.234773 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
I20260809 16:29:49.235597 5370 heartbeater.cc:347] Connected to a master server at 127.30.23.190:33281
I20260809 16:29:49.235713 5370 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:49.235883 5370 heartbeater.cc:511] Master 127.30.23.190:33281 requested a full tablet report, sending...
I20260809 16:29:49.236419 5047 ts_manager.cc:194] Registered new tserver with Master: fa2eaeb699c7412b99455b5ca0c494cc (127.30.23.129:41043)
I20260809 16:29:49.237112 5047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:54769
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:29:49.260165 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33281
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:49.344489 5376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:49.344709 5376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:49.344734 5376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:49.345956 5376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:49.345999 5376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:49.346027 5376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:29:49.347592 5376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:49.347882 5376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:49.348129 5376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:49.353609 5394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:49.353609 5386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:49.353609 5389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:49.353849 5376 server_base.cc:1034] running on GCE node
I20260809 16:29:49.354216 5376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:49.354496 5376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:49.355682 5376 hybrid_clock.cc:648] HybridClock initialized: now 1786292989355671 us; error 47 us; skew 500 ppm
Aug 09 16:29:49 dist-test-slave-j2vl krb5kdc[4967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292989, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:49.358650 5376 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:29:49.359061 5376 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:49.359112 5376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:49.359215 5376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:49.360278 5376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "fc0ff38800174a76964ea459865c0e9c"
format_stamp: "Formatted at 2026-08-09 16:29:49 on dist-test-slave-j2vl"
server_key: "b708eaed1bed72a8504645607d2c7dfe"
server_key_iv: "0b12acf0a2c0f8d9fd785a5325291504"
server_key_version: "encryptionkey@0"
I20260809 16:29:49.360601 5376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "fc0ff38800174a76964ea459865c0e9c"
format_stamp: "Formatted at 2026-08-09 16:29:49 on dist-test-slave-j2vl"
server_key: "b708eaed1bed72a8504645607d2c7dfe"
server_key_iv: "0b12acf0a2c0f8d9fd785a5325291504"
server_key_version: "encryptionkey@0"
I20260809 16:29:49.362183 5376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:29:49.363072 5400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:49.363298 5376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:49.363394 5376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "fc0ff38800174a76964ea459865c0e9c"
format_stamp: "Formatted at 2026-08-09 16:29:49 on dist-test-slave-j2vl"
server_key: "b708eaed1bed72a8504645607d2c7dfe"
server_key_iv: "0b12acf0a2c0f8d9fd785a5325291504"
server_key_version: "encryptionkey@0"
I20260809 16:29:49.363453 5376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:49.380316 5376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:49.382992 5376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:49.383117 5376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:49.383416 5376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:49.383769 5376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:49.383806 5376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:49.383829 5376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:49.383842 5376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:49.390301 5376 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:33603
I20260809 16:29:49.390376 5583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:33603 every 8 connection(s)
I20260809 16:29:49.390637 5376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:29:49.396034 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 5376
I20260809 16:29:49.396185 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:29:49.396435 30814 external_mini_cluster.cc:1428] Setting key 9d22c0c731c758827a6c6f4a570657d4
Aug 09 16:29:49 dist-test-slave-j2vl krb5kdc[4967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292989, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:49.402309 5107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.391418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:59423 (local address 127.30.23.190:33281)
0809 16:29:49.391525 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:49.391527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:49.391766 (+ 239us) server_negotiation.cc:407] Connection header received
0809 16:29:49.392278 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:49.392280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:49.392299 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:49.392372 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:49.393650 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.394183 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.395326 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.395531 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.395602 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.398347 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:49.398350 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:49.398350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:49.398367 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:49.400180 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:49.400812 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:49.400814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:49.400814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:49.400887 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:49.401284 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:49.401285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:49.401286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:49.401484 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:49.401552 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:49.402149 (+ 597us) server_negotiation.cc:299] Negotiation successful
0809 16:29:49.402240 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 16:29:49.402683 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.391386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33281 (local address 127.30.23.130:59423)
0809 16:29:49.391589 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:49.391634 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:49.392153 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:49.392472 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:49.392475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:49.392757 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:49.393285 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.393291 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.394299 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.394300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.395141 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.395144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.395636 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.395637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.395756 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.396417 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:49.396427 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:49.398250 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:49.400296 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:49.400306 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:49.400309 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:49.400689 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:49.400968 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:49.400968 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:49.400969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:49.401156 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:49.401626 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:49.401633 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:49.401828 (+ 195us) client_negotiation.cc:769] Sending connection context
0809 16:29:49.402081 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 16:29:49.402368 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":114,"threads_started":1}
I20260809 16:29:49.403234 5586 heartbeater.cc:347] Connected to a master server at 127.30.23.190:33281
I20260809 16:29:49.403362 5586 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:49.403693 5586 heartbeater.cc:511] Master 127.30.23.190:33281 requested a full tablet report, sending...
I20260809 16:29:49.404177 5047 ts_manager.cc:194] Registered new tserver with Master: fc0ff38800174a76964ea459865c0e9c (127.30.23.130:33603)
I20260809 16:29:49.404578 5047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:59423
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:29:49.429771 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33281
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:49.519910 5595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:49.520181 5595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:49.520210 5595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:49.521915 5595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:49.521982 5595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:49.522017 5595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:29:49.523741 5595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:49.524019 5595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:49.524282 5595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:49.529889 5604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:49.530019 5605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:49.529925 5608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:49.530192 5595 server_base.cc:1034] running on GCE node
I20260809 16:29:49.530472 5595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:49.530712 5595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:49.531873 5595 hybrid_clock.cc:648] HybridClock initialized: now 1786292989531867 us; error 43 us; skew 500 ppm
Aug 09 16:29:49 dist-test-slave-j2vl krb5kdc[4967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292989, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:49.534821 5595 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:29:49.535300 5595 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:49.535359 5595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:49.535558 5595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:49.536772 5595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9"
format_stamp: "Formatted at 2026-08-09 16:29:49 on dist-test-slave-j2vl"
server_key: "78402a9ab7facdcc03d0b0dd709fa4ea"
server_key_iv: "ab19703e2be25e5ac30e3582dd4e19d6"
server_key_version: "encryptionkey@0"
I20260809 16:29:49.537194 5595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9"
format_stamp: "Formatted at 2026-08-09 16:29:49 on dist-test-slave-j2vl"
server_key: "78402a9ab7facdcc03d0b0dd709fa4ea"
server_key_iv: "ab19703e2be25e5ac30e3582dd4e19d6"
server_key_version: "encryptionkey@0"
I20260809 16:29:49.538760 5595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:49.539786 5619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:49.540078 5595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:49.540174 5595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9"
format_stamp: "Formatted at 2026-08-09 16:29:49 on dist-test-slave-j2vl"
server_key: "78402a9ab7facdcc03d0b0dd709fa4ea"
server_key_iv: "ab19703e2be25e5ac30e3582dd4e19d6"
server_key_version: "encryptionkey@0"
I20260809 16:29:49.540283 5595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:49.566200 5595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:49.568800 5595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:49.568930 5595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:49.569139 5595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:49.569540 5595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:49.569581 5595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:49.569607 5595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:49.569653 5595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:49.576306 5595 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:37909
I20260809 16:29:49.576341 5787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:37909 every 8 connection(s)
I20260809 16:29:49.576676 5595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:29:49 dist-test-slave-j2vl krb5kdc[4967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292989, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:49.586746 5107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.577480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:41771 (local address 127.30.23.190:33281)
0809 16:29:49.577558 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:49.577561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:49.577781 (+ 220us) server_negotiation.cc:407] Connection header received
0809 16:29:49.578276 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:49.578277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:49.578288 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:49.578345 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:49.579361 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.579855 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.580832 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.581020 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.581075 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.583518 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:49.583520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:49.583521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:49.583529 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:49.585141 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:49.585709 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:49.585710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:49.585711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:49.585774 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:49.586022 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:49.586023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:49.586023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:49.586149 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:49.586206 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:49.586576 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 16:29:49.586664 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:29:49.586967 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 5595
I20260809 16:29:49.587076 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:29:49.587124 5791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.577439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33281 (local address 127.30.23.131:41771)
0809 16:29:49.577654 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:49.577693 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:49.578184 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:49.578414 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:49.578417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:49.578770 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:49.579276 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.579281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.579944 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.579945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.580754 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.580756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.581097 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.581098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.581201 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.581859 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:49.581868 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:49.583369 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:49.585274 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:49.585282 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:49.585284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:49.585629 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:49.585849 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:49.585849 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:49.585850 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:49.585955 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:49.586254 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:49.586261 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:49.586399 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 16:29:49.586492 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 16:29:49.586698 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":113,"threads_started":1}
I20260809 16:29:49.587283 30814 external_mini_cluster.cc:1428] Setting key 526a00b09dd0e7e629fa9af75ab58ec0
I20260809 16:29:49.587807 5788 heartbeater.cc:347] Connected to a master server at 127.30.23.190:33281
I20260809 16:29:49.587939 5788 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:49.588109 5788 heartbeater.cc:511] Master 127.30.23.190:33281 requested a full tablet report, sending...
I20260809 16:29:49.588511 5047 ts_manager.cc:194] Registered new tserver with Master: dfacd3d6f55448c6b98c3c6ea6292cf9 (127.30.23.131:37909)
I20260809 16:29:49.588871 5047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:41771
I20260809 16:29:49.599786 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:29:49.607637 5107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.601157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60708 (local address 127.30.23.190:33281)
0809 16:29:49.601207 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:49.601209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:49.601322 (+ 113us) server_negotiation.cc:407] Connection header received
0809 16:29:49.601451 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:49.601451 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:49.601462 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:49.601510 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:49.602077 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.602599 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.603352 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.603546 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.603615 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.604919 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:49.604921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:49.604921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:49.604929 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:49.606340 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:49.606745 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:49.606746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:49.606746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:49.606804 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:49.607041 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:49.607041 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:49.607042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:49.607195 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:49.607276 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:49.607496 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 16:29:49.607570 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:29:49.609617 5047 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:29:49.611286 5047 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:29:49.618568 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.614000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:33603 (local address 127.0.0.1:44276)
0809 16:29:49.614250 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:49.614262 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:49.614332 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:49.614904 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:49.614908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:49.614910 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:49.614990 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.614992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.616387 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.616388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.617506 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.617508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.618213 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.618215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.618301 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.618315 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:29:49.618427 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 16:29:49.618488 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
I20260809 16:29:49.618577 5812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.614296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44276 (local address 127.30.23.130:33603)
0809 16:29:49.614741 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:49.614745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:49.614754 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:29:49.614790 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:49.614792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:49.614806 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:49.614888 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:49.615077 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.616267 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.617595 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.618103 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.618210 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.618276 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:49.618446 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 16:29:49.618488 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":85,"threads_started":1}
I20260809 16:29:49.618886 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.614122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.30.23.131:37909)
0809 16:29:49.615029 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:49.615034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:49.615045 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:29:49.615084 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:49.615086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:49.615101 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:49.615195 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:49.615289 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.616334 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.617479 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.618243 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.618305 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.618357 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:49.618752 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 16:29:49.618797 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":871,"thread_start_us":89,"threads_started":1}
I20260809 16:29:49.618906 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.614505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45250 (local address 127.30.23.129:41043)
0809 16:29:49.614670 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:49.614676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:49.614688 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:29:49.614775 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:49.614777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:49.614794 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:49.614885 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:49.615148 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.615941 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.617450 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.618199 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.618293 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.618357 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:49.618642 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 16:29:49.618687 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260809 16:29:49.619061 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.613999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:41043 (local address 127.0.0.1:45250)
0809 16:29:49.614212 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:49.614256 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:49.614331 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:49.615025 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:49.615026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:49.615028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:49.615081 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.615082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.616050 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.616051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.617370 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.617373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.618398 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.618398 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.618479 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.618486 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:29:49.618537 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:29:49.618594 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":8,"thread_start_us":59,"threads_started":1}
I20260809 16:29:49.618678 5808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.614022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:37909 (local address 127.0.0.1:46112)
0809 16:29:49.614296 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:49.614305 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:49.614379 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:49.615196 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:49.615197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:49.615199 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:49.615244 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.615245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.616441 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.616443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.617330 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.617333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.618396 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.618398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.618479 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.618486 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:29:49.618537 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:29:49.618594 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":10,"thread_start_us":111,"threads_started":1}
I20260809 16:29:49.619606 5482 tablet_service.cc:1467] Processing CreateTablet for tablet e868f588a081473c85ae88fa8cc97ee1 (DEFAULT_TABLE table=test-table [id=3c1bd569184d4d49bbc0fe894c54fc45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:49.619813 5264 tablet_service.cc:1467] Processing CreateTablet for tablet e868f588a081473c85ae88fa8cc97ee1 (DEFAULT_TABLE table=test-table [id=3c1bd569184d4d49bbc0fe894c54fc45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:49.620036 5482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e868f588a081473c85ae88fa8cc97ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:49.620138 5264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e868f588a081473c85ae88fa8cc97ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:49.620083 5687 tablet_service.cc:1467] Processing CreateTablet for tablet e868f588a081473c85ae88fa8cc97ee1 (DEFAULT_TABLE table=test-table [id=3c1bd569184d4d49bbc0fe894c54fc45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:49.620425 5687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e868f588a081473c85ae88fa8cc97ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:49.623270 5815 tablet_bootstrap.cc:492] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9: Bootstrap starting.
I20260809 16:29:49.623924 5818 tablet_bootstrap.cc:492] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c: Bootstrap starting.
I20260809 16:29:49.624207 5815 tablet_bootstrap.cc:654] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:49.624554 5818 tablet_bootstrap.cc:654] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:49.624825 5815 log.cc:824] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:49.625187 5818 log.cc:824] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:49.625204 5816 tablet_bootstrap.cc:492] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc: Bootstrap starting.
I20260809 16:29:49.625723 5815 tablet_bootstrap.cc:492] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9: No bootstrap required, opened a new log
I20260809 16:29:49.625861 5815 ts_tablet_manager.cc:1397] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:29:49.625933 5816 tablet_bootstrap.cc:654] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:49.626134 5818 tablet_bootstrap.cc:492] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c: No bootstrap required, opened a new log
I20260809 16:29:49.626247 5818 ts_tablet_manager.cc:1397] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:29:49.626535 5816 log.cc:824] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:49.627287 5816 tablet_bootstrap.cc:492] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc: No bootstrap required, opened a new log
I20260809 16:29:49.627395 5816 ts_tablet_manager.cc:1397] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:29:49.628139 5815 raft_consensus.cc:348] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0ff38800174a76964ea459865c0e9c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 33603 } } peers { permanent_uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 37909 } } peers { permanent_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41043 } }
I20260809 16:29:49.628350 5815 raft_consensus.cc:374] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:49.628393 5815 raft_consensus.cc:729] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfacd3d6f55448c6b98c3c6ea6292cf9, State: Initialized, Role: FOLLOWER
I20260809 16:29:49.628568 5815 consensus_queue.cc:260] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0ff38800174a76964ea459865c0e9c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 33603 } } peers { permanent_uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 37909 } } peers { permanent_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41043 } }
I20260809 16:29:49.628494 5818 raft_consensus.cc:348] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0ff38800174a76964ea459865c0e9c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 33603 } } peers { permanent_uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 37909 } } peers { permanent_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41043 } }
I20260809 16:29:49.628674 5818 raft_consensus.cc:374] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:49.628750 5818 raft_consensus.cc:729] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0ff38800174a76964ea459865c0e9c, State: Initialized, Role: FOLLOWER
I20260809 16:29:49.628924 5815 ts_tablet_manager.cc:1428] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:29:49.628942 5788 heartbeater.cc:503] Master 127.30.23.190:33281 was elected leader, sending a full tablet report...
I20260809 16:29:49.628969 5818 consensus_queue.cc:260] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0ff38800174a76964ea459865c0e9c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 33603 } } peers { permanent_uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 37909 } } peers { permanent_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41043 } }
I20260809 16:29:49.629185 5816 raft_consensus.cc:348] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0ff38800174a76964ea459865c0e9c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 33603 } } peers { permanent_uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 37909 } } peers { permanent_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41043 } }
I20260809 16:29:49.629295 5586 heartbeater.cc:503] Master 127.30.23.190:33281 was elected leader, sending a full tablet report...
I20260809 16:29:49.629359 5816 raft_consensus.cc:374] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:49.629312 5818 ts_tablet_manager.cc:1428] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:29:49.629406 5816 raft_consensus.cc:729] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa2eaeb699c7412b99455b5ca0c494cc, State: Initialized, Role: FOLLOWER
I20260809 16:29:49.629514 5816 consensus_queue.cc:260] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0ff38800174a76964ea459865c0e9c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 33603 } } peers { permanent_uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 37909 } } peers { permanent_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41043 } }
I20260809 16:29:49.629980 5370 heartbeater.cc:503] Master 127.30.23.190:33281 was elected leader, sending a full tablet report...
I20260809 16:29:49.630086 5816 ts_tablet_manager.cc:1428] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260809 16:29:49.641192 5590 tablet.cc:2369] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:29:49.725168 5372 tablet.cc:2369] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:29:49.749585 5826 raft_consensus.cc:482] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:29:49.749742 5826 raft_consensus.cc:504] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0ff38800174a76964ea459865c0e9c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 33603 } } peers { permanent_uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 37909 } } peers { permanent_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41043 } }
I20260809 16:29:49.750125 5826 leader_election.cc:288] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc0ff38800174a76964ea459865c0e9c (127.30.23.130:33603), dfacd3d6f55448c6b98c3c6ea6292cf9 (127.30.23.131:37909)
I20260809 16:29:49.753505 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.750557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:34625 (local address 127.30.23.131:37909)
0809 16:29:49.750621 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:49.750623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:49.750737 (+ 114us) server_negotiation.cc:407] Connection header received
0809 16:29:49.750821 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:49.750822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:49.750833 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:49.750892 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:49.751084 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.751595 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.752688 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.753099 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.753188 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.753199 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:49.753391 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 16:29:49.753430 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:29:49.753533 5827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.750418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:37909 (local address 127.30.23.129:34625)
0809 16:29:49.750642 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:49.750653 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:49.750757 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:49.750902 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:49.750903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:49.750905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:49.751003 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.751005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.751686 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.751687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.752578 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.752580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.753189 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.753190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.753269 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.753273 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:49.753343 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 16:29:49.753408 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":18,"spinlock_wait_cycles":12160,"thread_start_us":104,"threads_started":1}
I20260809 16:29:49.753522 5812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.750500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:45843 (local address 127.30.23.130:33603)
0809 16:29:49.750552 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:49.750563 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:49.750766 (+ 203us) server_negotiation.cc:407] Connection header received
0809 16:29:49.750840 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:49.750841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:49.750858 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:49.750923 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:49.751096 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.751577 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.752681 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.753118 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.753207 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.753216 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:49.753411 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 16:29:49.753450 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:29:49.753548 5828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.750425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:33603 (local address 127.30.23.129:45843)
0809 16:29:49.750672 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:49.750681 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:49.750771 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:49.750927 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:49.750928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:49.750930 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:49.751020 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.751022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.751691 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.751693 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.752579 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:49.752582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.753207 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.753209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:49.753293 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.753297 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:49.753401 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 16:29:49.753445 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":76,"thread_start_us":100,"threads_started":1}
I20260809 16:29:49.754017 5719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e868f588a081473c85ae88fa8cc97ee1" candidate_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9" is_pre_election: true
I20260809 16:29:49.754204 5719 raft_consensus.cc:2454] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa2eaeb699c7412b99455b5ca0c494cc in term 0.
I20260809 16:29:49.754196 5512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e868f588a081473c85ae88fa8cc97ee1" candidate_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0ff38800174a76964ea459865c0e9c" is_pre_election: true
I20260809 16:29:49.754336 5512 raft_consensus.cc:2454] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa2eaeb699c7412b99455b5ca0c494cc in term 0.
I20260809 16:29:49.754559 5178 leader_election.cc:302] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfacd3d6f55448c6b98c3c6ea6292cf9, fa2eaeb699c7412b99455b5ca0c494cc; no voters:
I20260809 16:29:49.754729 5826 raft_consensus.cc:2781] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:29:49.754792 5826 raft_consensus.cc:482] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:29:49.754827 5826 raft_consensus.cc:3037] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:49.755563 5826 raft_consensus.cc:504] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0ff38800174a76964ea459865c0e9c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 33603 } } peers { permanent_uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 37909 } } peers { permanent_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41043 } }
I20260809 16:29:49.755740 5826 leader_election.cc:288] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [CANDIDATE]: Term 1 election: Requested vote from peers fc0ff38800174a76964ea459865c0e9c (127.30.23.130:33603), dfacd3d6f55448c6b98c3c6ea6292cf9 (127.30.23.131:37909)
I20260809 16:29:49.755906 5512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e868f588a081473c85ae88fa8cc97ee1" candidate_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0ff38800174a76964ea459865c0e9c"
I20260809 16:29:49.755985 5719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e868f588a081473c85ae88fa8cc97ee1" candidate_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9"
I20260809 16:29:49.756008 5512 raft_consensus.cc:3037] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:49.756058 5719 raft_consensus.cc:3037] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:49.756987 5512 raft_consensus.cc:2454] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa2eaeb699c7412b99455b5ca0c494cc in term 1.
I20260809 16:29:49.756987 5719 raft_consensus.cc:2454] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa2eaeb699c7412b99455b5ca0c494cc in term 1.
I20260809 16:29:49.757231 5180 leader_election.cc:302] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa2eaeb699c7412b99455b5ca0c494cc, fc0ff38800174a76964ea459865c0e9c; no voters:
I20260809 16:29:49.757405 5826 raft_consensus.cc:2781] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:49.757618 5826 raft_consensus.cc:686] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [term 1 LEADER]: Becoming Leader. State: Replica: fa2eaeb699c7412b99455b5ca0c494cc, State: Running, Role: LEADER
I20260809 16:29:49.757710 5826 consensus_queue.cc:237] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0ff38800174a76964ea459865c0e9c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 33603 } } peers { permanent_uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 37909 } } peers { permanent_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41043 } }
I20260809 16:29:49.758714 5046 catalog_manager.cc:5495] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc reported cstate change: term changed from 0 to 1, leader changed from <none> to fa2eaeb699c7412b99455b5ca0c494cc (127.30.23.129). New cstate: current_term: 1 leader_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc0ff38800174a76964ea459865c0e9c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 33603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa2eaeb699c7412b99455b5ca0c494cc" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 41043 } health_report { overall_health: HEALTHY } } }
I20260809 16:29:49.766230 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.763788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.30.23.129:41043)
0809 16:29:49.763874 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:49.763876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:49.763886 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:29:49.763917 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:49.763918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:49.763935 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:49.764003 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:49.764159 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.764629 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.765460 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:49.765650 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:49.765710 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:49.765878 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:49.765978 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:49.766086 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:49.766101 (+ 15us) server_negotiation.cc:299] Negotiation successful
0809 16:29:49.766121 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 16:29:49.768240 5512 raft_consensus.cc:1261] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c [term 1 FOLLOWER]: Refusing update from remote peer fa2eaeb699c7412b99455b5ca0c494cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:49.768241 5719 raft_consensus.cc:1261] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9 [term 1 FOLLOWER]: Refusing update from remote peer fa2eaeb699c7412b99455b5ca0c494cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:49.768726 5831 consensus_queue.cc:1035] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfacd3d6f55448c6b98c3c6ea6292cf9" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:49.768872 5826 consensus_queue.cc:1035] T e868f588a081473c85ae88fa8cc97ee1 P fa2eaeb699c7412b99455b5ca0c494cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0ff38800174a76964ea459865c0e9c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 33603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:49.771633 5834 mvcc.cc:205] Tried to move back new op lower bound from 7316656086087925760 to 7316656086048276480. Current Snapshot: MvccSnapshot[applied={T|T < 7316656086087925760}]
I20260809 16:29:49.772354 5837 mvcc.cc:205] Tried to move back new op lower bound from 7316656086087925760 to 7316656086048276480. Current Snapshot: MvccSnapshot[applied={T|T < 7316656086087925760 or (T in {7316656086087925760})}]
I20260809 16:29:49.774976 5047 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:49.775089 5047 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:49.775996 5047 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1b88da8686d44c691106d241db34819: Sending DeleteTablet for 3 replicas of tablet e868f588a081473c85ae88fa8cc97ee1
I20260809 16:29:49.776387 5687 tablet_service.cc:1514] Processing DeleteTablet for tablet e868f588a081473c85ae88fa8cc97ee1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:49 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:46112
I20260809 16:29:49.776382 5264 tablet_service.cc:1514] Processing DeleteTablet for tablet e868f588a081473c85ae88fa8cc97ee1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:49 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:45250
I20260809 16:29:49.776551 5482 tablet_service.cc:1514] Processing DeleteTablet for tablet e868f588a081473c85ae88fa8cc97ee1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:49 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:44276
I20260809 16:29:49.776897 5851 tablet_replica.cc:330] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c: stopping tablet replica
I20260809 16:29:49.776891 5849 tablet_replica.cc:330] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9: stopping tablet replica
I20260809 16:29:49.777060 5851 raft_consensus.cc:2229] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:49.777182 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 5149
I20260809 16:29:49.777185 5849 raft_consensus.cc:2229] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:49.777221 5851 raft_consensus.cc:2258] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:49.782336 5849 raft_consensus.cc:2258] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:49.784271 5849 ts_tablet_manager.cc:1905] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:49.786690 5849 ts_tablet_manager.cc:1918] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:29:49.786770 5849 log.cc:1196] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/e868f588a081473c85ae88fa8cc97ee1
I20260809 16:29:49.787014 5849 ts_tablet_manager.cc:1939] T e868f588a081473c85ae88fa8cc97ee1 P dfacd3d6f55448c6b98c3c6ea6292cf9: Deleting consensus metadata
I20260809 16:29:49.787626 5015 catalog_manager.cc:4841] TS dfacd3d6f55448c6b98c3c6ea6292cf9 (127.30.23.131:37909): tablet e868f588a081473c85ae88fa8cc97ee1 (table test-table [id=3c1bd569184d4d49bbc0fe894c54fc45]) successfully deleted
I20260809 16:29:49.792189 5851 ts_tablet_manager.cc:1905] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:49.793641 5851 ts_tablet_manager.cc:1918] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:29:49.793740 5851 log.cc:1196] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/e868f588a081473c85ae88fa8cc97ee1
I20260809 16:29:49.793999 5851 ts_tablet_manager.cc:1939] T e868f588a081473c85ae88fa8cc97ee1 P fc0ff38800174a76964ea459865c0e9c: Deleting consensus metadata
I20260809 16:29:49.794497 5020 catalog_manager.cc:4841] TS fc0ff38800174a76964ea459865c0e9c (127.30.23.130:33603): tablet e868f588a081473c85ae88fa8cc97ee1 (table test-table [id=3c1bd569184d4d49bbc0fe894c54fc45]) successfully deleted
W20260809 16:29:49.795940 5021 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:41043 (error 108)
I20260809 16:29:49.796298 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 5376
I20260809 16:29:49.801005 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.796380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:41043 (local address 127.0.0.1:45258)
0809 16:29:49.800912 (+ 4532us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:49.800972 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:41043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4502}
W20260809 16:29:49.802306 5021 catalog_manager.cc:4584] TS fa2eaeb699c7412b99455b5ca0c494cc (127.30.23.129:41043): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e868f588a081473c85ae88fa8cc97ee1: Network error: Client connection negotiation failed: client connection to 127.30.23.129:41043: connect: Connection refused (error 111)
I20260809 16:29:49.808202 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 5595
I20260809 16:29:49.816584 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 4988
2026-08-09T16:29:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2370 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T16:29:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:49Z Disabled control of system clock
I20260809 16:29:49.856702 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:44865
--webserver_interface=127.30.23.190
--webserver_port=0
--builtin_ntp_servers=127.30.23.148:42937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:44865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 16:29:49.941802 5858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:49.942040 5858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:49.942063 5858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:49.943272 5858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:49.943320 5858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:49.943333 5858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:49.943346 5858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:49.943358 5858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:49.944765 5858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:42937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:44865
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:44865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:49.945060 5858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:49.945416 5858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:29:49.950660 5858 server_base.cc:1034] running on GCE node
W20260809 16:29:49.950677 5870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:49.950687 5873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:49.950825 5869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:49.951186 5858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:49.951510 5858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:49.952710 5858 hybrid_clock.cc:648] HybridClock initialized: now 1786292989952681 us; error 62 us; skew 500 ppm
I20260809 16:29:49.954221 5858 webserver.cc:460] Webserver started at http://127.30.23.190:37373/ using document root <none> and password file <none>
I20260809 16:29:49.954526 5858 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:49.954607 5858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:49.954854 5858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:49.956002 5858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "1e2dc8047b7a4dcbbbb1846c73b0918b"
format_stamp: "Formatted at 2026-08-09 16:29:49 on dist-test-slave-j2vl"
server_key: "cf107debbc71e8300a128df762435027"
server_key_iv: "0702f92a7a348cc784152be747518a64"
server_key_version: "encryptionkey@0"
I20260809 16:29:49.956384 5858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "1e2dc8047b7a4dcbbbb1846c73b0918b"
format_stamp: "Formatted at 2026-08-09 16:29:49 on dist-test-slave-j2vl"
server_key: "cf107debbc71e8300a128df762435027"
server_key_iv: "0702f92a7a348cc784152be747518a64"
server_key_version: "encryptionkey@0"
I20260809 16:29:49.957875 5858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:29:49.958803 5881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:49.959082 5858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:49.959201 5858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "1e2dc8047b7a4dcbbbb1846c73b0918b"
format_stamp: "Formatted at 2026-08-09 16:29:49 on dist-test-slave-j2vl"
server_key: "cf107debbc71e8300a128df762435027"
server_key_iv: "0702f92a7a348cc784152be747518a64"
server_key_version: "encryptionkey@0"
I20260809 16:29:49.959303 5858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:49.962956 5858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:49.963263 5858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:49.963419 5858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:49.967996 5858 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:44865
I20260809 16:29:49.968093 5964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:44865 every 8 connection(s)
I20260809 16:29:49.968459 5858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:49.969669 5965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:49.971791 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 5858
I20260809 16:29:49.971922 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:49.972121 30814 external_mini_cluster.cc:1428] Setting key e53a57c1965bc21a2038a7dd48697a0d
I20260809 16:29:49.973078 5965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b: Bootstrap starting.
I20260809 16:29:49.973785 5965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:49.974241 5965 log.cc:824] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:49.975155 5965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b: No bootstrap required, opened a new log
I20260809 16:29:49.975468 5968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:49.972758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47758 (local address 127.30.23.190:44865)
0809 16:29:49.972962 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:49.972969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:49.972995 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:29:49.973675 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:49.973684 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:49.973730 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:49.973864 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:49.974729 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:49.974733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:49.974734 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:49.974739 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:49.974798 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:49.974813 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:49.974896 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:49.975224 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 16:29:49.975291 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260809 16:29:49.976814 5965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2dc8047b7a4dcbbbb1846c73b0918b" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44865 } }
I20260809 16:29:49.976984 5965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:49.977035 5965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2dc8047b7a4dcbbbb1846c73b0918b, State: Initialized, Role: FOLLOWER
I20260809 16:29:49.977190 5965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2dc8047b7a4dcbbbb1846c73b0918b" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44865 } }
I20260809 16:29:49.977669 5965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:49.977722 5965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:49.977777 5965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:49.978364 5965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2dc8047b7a4dcbbbb1846c73b0918b" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44865 } }
I20260809 16:29:49.978489 5965 leader_election.cc:302] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e2dc8047b7a4dcbbbb1846c73b0918b; no voters:
I20260809 16:29:49.978681 5965 leader_election.cc:288] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:49.978786 5970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:49.978968 5970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [term 1 LEADER]: Becoming Leader. State: Replica: 1e2dc8047b7a4dcbbbb1846c73b0918b, State: Running, Role: LEADER
I20260809 16:29:49.979050 5965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:49.979116 5970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2dc8047b7a4dcbbbb1846c73b0918b" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44865 } }
I20260809 16:29:49.979557 5972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e2dc8047b7a4dcbbbb1846c73b0918b. Latest consensus state: current_term: 1 leader_uuid: "1e2dc8047b7a4dcbbbb1846c73b0918b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2dc8047b7a4dcbbbb1846c73b0918b" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44865 } } }
I20260809 16:29:49.979545 5971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e2dc8047b7a4dcbbbb1846c73b0918b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2dc8047b7a4dcbbbb1846c73b0918b" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44865 } } }
I20260809 16:29:49.979679 5972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:49.979683 5971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:49.980005 5979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:49.980540 5979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:49.982170 5979 catalog_manager.cc:1347] Generated new cluster ID: 4fc55a27e6fe41e7a3c457471e9954b3
I20260809 16:29:49.982226 5979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:29:50.000801 5979 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:50.001029 5979 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:50.017663 5979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b: Generated new TSK 0
I20260809 16:29:50.017904 5979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:29:50.026358 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44865
--builtin_ntp_servers=127.30.23.148:42937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 16:29:50.154778 6000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:50.155000 6000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:50.155026 6000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:50.156251 6000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:50.156289 6000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:50.156318 6000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:29:50.157829 6000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:42937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:50.158046 6000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:50.158324 6000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:50.163944 6010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:50.163916 6007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:50.164348 6000 server_base.cc:1034] running on GCE node
W20260809 16:29:50.163964 6008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:50.164687 6000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:50.164985 6000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:50.166121 6000 hybrid_clock.cc:648] HybridClock initialized: now 1786292990166095 us; error 39 us; skew 500 ppm
I20260809 16:29:50.168032 6000 webserver.cc:460] Webserver started at http://127.30.23.129:35449/ using document root <none> and password file <none>
I20260809 16:29:50.168366 6000 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:50.168444 6000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:50.168599 6000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:50.170253 6000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7"
format_stamp: "Formatted at 2026-08-09 16:29:50 on dist-test-slave-j2vl"
server_key: "dbeaa20e5bcb85fa2895396bcd11683a"
server_key_iv: "c1bad97b949fa1b6102c682c06cc7129"
server_key_version: "encryptionkey@0"
I20260809 16:29:50.170719 6000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7"
format_stamp: "Formatted at 2026-08-09 16:29:50 on dist-test-slave-j2vl"
server_key: "dbeaa20e5bcb85fa2895396bcd11683a"
server_key_iv: "c1bad97b949fa1b6102c682c06cc7129"
server_key_version: "encryptionkey@0"
I20260809 16:29:50.173780 6000 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 16:29:50.174989 6020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:50.175211 6000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:50.175324 6000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7"
format_stamp: "Formatted at 2026-08-09 16:29:50 on dist-test-slave-j2vl"
server_key: "dbeaa20e5bcb85fa2895396bcd11683a"
server_key_iv: "c1bad97b949fa1b6102c682c06cc7129"
server_key_version: "encryptionkey@0"
I20260809 16:29:50.175441 6000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:50.179874 6000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:50.180177 6000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:50.180383 6000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:50.180605 6000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:50.180943 6000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:50.180992 6000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:50.181031 6000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:50.181058 6000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:50.186937 6000 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:37659
I20260809 16:29:50.187044 6203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:37659 every 8 connection(s)
I20260809 16:29:50.187368 6000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:29:50.190526 5968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.188190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:33525 (local address 127.30.23.190:44865)
0809 16:29:50.188263 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:50.188265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:50.188529 (+ 264us) server_negotiation.cc:407] Connection header received
0809 16:29:50.189010 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:50.189012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:50.189018 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:50.189085 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:50.190292 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:50.190294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:50.190294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:50.190299 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:50.190314 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:50.190318 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:50.190389 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:50.190449 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 16:29:50.190463 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:29:50.190682 6207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.188188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44865 (local address 127.30.23.129:33525)
0809 16:29:50.188399 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:50.188439 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:50.188898 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:50.189135 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:50.189138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:50.189146 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:50.190090 (+ 944us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:29:50.190090 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:50.190118 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:29:50.190147 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:50.190385 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:50.190387 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:50.190389 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:29:50.190443 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 16:29:50.190491 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260809 16:29:50.191345 6204 heartbeater.cc:347] Connected to a master server at 127.30.23.190:44865
I20260809 16:29:50.191442 6204 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:50.191557 6204 heartbeater.cc:511] Master 127.30.23.190:44865 requested a full tablet report, sending...
I20260809 16:29:50.192123 5905 ts_manager.cc:194] Registered new tserver with Master: ff34d67c679b47c59ecdcf8d5e7e51f7 (127.30.23.129:37659)
I20260809 16:29:50.193908 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 6000
I20260809 16:29:50.194037 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:29:50.194247 30814 external_mini_cluster.cc:1428] Setting key f1c0882471e1afd002bf1341e73b4210
I20260809 16:29:50.195243 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44865
--builtin_ntp_servers=127.30.23.148:42937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 16:29:50.279349 6208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:50.279655 6208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:50.279700 6208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:50.281090 6208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:50.281164 6208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:50.281210 6208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:29:50.282684 6208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:42937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:50.282989 6208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:50.283315 6208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:50.288748 6217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:50.288762 6216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:50.288723 6223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:50.289131 6208 server_base.cc:1034] running on GCE node
I20260809 16:29:50.289408 6208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:50.289642 6208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:50.290781 6208 hybrid_clock.cc:648] HybridClock initialized: now 1786292990290769 us; error 38 us; skew 500 ppm
I20260809 16:29:50.292130 6208 webserver.cc:460] Webserver started at http://127.30.23.130:38177/ using document root <none> and password file <none>
I20260809 16:29:50.292376 6208 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:50.292424 6208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:50.292577 6208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:50.293753 6208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "a0453e43f6864efaa2ca0e687d7e7937"
format_stamp: "Formatted at 2026-08-09 16:29:50 on dist-test-slave-j2vl"
server_key: "1812bacfa6c05ddf2623eac6ec5ac6a6"
server_key_iv: "3a9a76b521b20ae32ccd8f4617c8f3b4"
server_key_version: "encryptionkey@0"
I20260809 16:29:50.294126 6208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "a0453e43f6864efaa2ca0e687d7e7937"
format_stamp: "Formatted at 2026-08-09 16:29:50 on dist-test-slave-j2vl"
server_key: "1812bacfa6c05ddf2623eac6ec5ac6a6"
server_key_iv: "3a9a76b521b20ae32ccd8f4617c8f3b4"
server_key_version: "encryptionkey@0"
I20260809 16:29:50.296329 6208 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 16:29:50.297181 6231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:50.297452 6208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:50.297572 6208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "a0453e43f6864efaa2ca0e687d7e7937"
format_stamp: "Formatted at 2026-08-09 16:29:50 on dist-test-slave-j2vl"
server_key: "1812bacfa6c05ddf2623eac6ec5ac6a6"
server_key_iv: "3a9a76b521b20ae32ccd8f4617c8f3b4"
server_key_version: "encryptionkey@0"
I20260809 16:29:50.297659 6208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:50.301211 6208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:50.301515 6208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:50.301663 6208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:50.301890 6208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:50.302212 6208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:50.302246 6208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:50.302301 6208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:50.302330 6208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:50.308256 6208 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:46791
I20260809 16:29:50.308337 6437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:46791 every 8 connection(s)
I20260809 16:29:50.308689 6208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:29:50.310879 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 6208
I20260809 16:29:50.310997 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:29:50.311223 30814 external_mini_cluster.cc:1428] Setting key 323890e58cea77f50c09c0ecc670ec8c
I20260809 16:29:50.312103 5968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.309481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:48557 (local address 127.30.23.190:44865)
0809 16:29:50.309555 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:50.309557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:50.309816 (+ 259us) server_negotiation.cc:407] Connection header received
0809 16:29:50.310347 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:50.310349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:50.310356 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:50.310400 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:50.311836 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:50.311838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:50.311838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:50.311842 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:50.311857 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:50.311861 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:50.311914 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:50.312015 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 16:29:50.312033 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:29:50.312309 6442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.309492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44865 (local address 127.30.23.130:48557)
0809 16:29:50.309694 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:50.309728 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:50.310230 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:50.310468 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:50.310472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:50.310481 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:50.311652 (+ 1171us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:29:50.311652 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:50.311711 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:29:50.311745 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:50.311934 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:50.311940 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:50.311943 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:29:50.312024 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:29:50.312088 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":73,"threads_started":1}
I20260809 16:29:50.312525 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44865
--builtin_ntp_servers=127.30.23.148:42937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 16:29:50.314409 6440 heartbeater.cc:347] Connected to a master server at 127.30.23.190:44865
I20260809 16:29:50.314514 6440 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:50.314684 6440 heartbeater.cc:511] Master 127.30.23.190:44865 requested a full tablet report, sending...
I20260809 16:29:50.315135 5905 ts_manager.cc:194] Registered new tserver with Master: a0453e43f6864efaa2ca0e687d7e7937 (127.30.23.130:46791)
W20260809 16:29:50.395617 6443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:50.395833 6443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:50.395857 6443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:50.397034 6443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:50.397080 6443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:50.397107 6443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:29:50.398533 6443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:42937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:50.398797 6443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:50.399132 6443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:50.404734 6455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:50.404729 6451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:50.404867 6443 server_base.cc:1034] running on GCE node
W20260809 16:29:50.404752 6452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:50.405330 6443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:50.405738 6443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:50.406982 6443 hybrid_clock.cc:648] HybridClock initialized: now 1786292990406904 us; error 95 us; skew 500 ppm
I20260809 16:29:50.408645 6443 webserver.cc:460] Webserver started at http://127.30.23.131:44835/ using document root <none> and password file <none>
I20260809 16:29:50.408938 6443 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:50.409015 6443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:50.409178 6443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:50.410410 6443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "0f5b0be24c734bb294b5c083bfd4ea1e"
format_stamp: "Formatted at 2026-08-09 16:29:50 on dist-test-slave-j2vl"
server_key: "cce4960c599e75dc3b73e1feba4dff02"
server_key_iv: "59fdb10729fbd81bf2a10fa653567df1"
server_key_version: "encryptionkey@0"
I20260809 16:29:50.410802 6443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "0f5b0be24c734bb294b5c083bfd4ea1e"
format_stamp: "Formatted at 2026-08-09 16:29:50 on dist-test-slave-j2vl"
server_key: "cce4960c599e75dc3b73e1feba4dff02"
server_key_iv: "59fdb10729fbd81bf2a10fa653567df1"
server_key_version: "encryptionkey@0"
I20260809 16:29:50.412238 6443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:29:50.413064 6464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:50.413262 6443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:50.413395 6443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "0f5b0be24c734bb294b5c083bfd4ea1e"
format_stamp: "Formatted at 2026-08-09 16:29:50 on dist-test-slave-j2vl"
server_key: "cce4960c599e75dc3b73e1feba4dff02"
server_key_iv: "59fdb10729fbd81bf2a10fa653567df1"
server_key_version: "encryptionkey@0"
I20260809 16:29:50.413489 6443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:50.417106 6443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:50.417460 6443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:50.417608 6443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:50.417840 6443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:50.418171 6443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:50.418236 6443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:50.418277 6443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:50.418305 6443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:50.424247 6443 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:34401
I20260809 16:29:50.424289 6653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:34401 every 8 connection(s)
I20260809 16:29:50.424621 6443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:29:50.427837 5968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.425379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:54751 (local address 127.30.23.190:44865)
0809 16:29:50.425428 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:50.425430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:50.425701 (+ 271us) server_negotiation.cc:407] Connection header received
0809 16:29:50.426226 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:50.426228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:50.426239 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:50.426291 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:50.427550 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:50.427551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:50.427552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:50.427556 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:50.427573 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:50.427577 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:50.427648 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:50.427743 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 16:29:50.427759 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:29:50.428035 6657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.425315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44865 (local address 127.30.23.131:54751)
0809 16:29:50.425581 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:50.425613 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:50.426105 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:50.426372 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:50.426375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:50.426383 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:50.427380 (+ 997us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:29:50.427380 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:50.427410 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:29:50.427439 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:50.427647 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:50.427654 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:50.427657 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:29:50.427754 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 16:29:50.427819 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":161,"threads_started":1}
I20260809 16:29:50.428573 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 6443
I20260809 16:29:50.428706 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:29:50.428716 6655 heartbeater.cc:347] Connected to a master server at 127.30.23.190:44865
I20260809 16:29:50.428831 6655 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:50.428961 6655 heartbeater.cc:511] Master 127.30.23.190:44865 requested a full tablet report, sending...
I20260809 16:29:50.428967 30814 external_mini_cluster.cc:1428] Setting key e6cebc2673b45ff61159cbd49067d528
I20260809 16:29:50.429391 5905 ts_manager.cc:194] Registered new tserver with Master: 0f5b0be24c734bb294b5c083bfd4ea1e (127.30.23.131:34401)
I20260809 16:29:50.430071 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:29:50.432592 5968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.431483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.30.23.190:44865)
0809 16:29:50.431547 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:50.431550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:50.431678 (+ 128us) server_negotiation.cc:407] Connection header received
0809 16:29:50.431716 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:50.431716 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:50.431721 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:50.431768 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:50.432304 (+ 536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:50.432305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:50.432306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:50.432309 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:50.432322 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:50.432325 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:50.432369 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:50.432481 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 16:29:50.432492 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:29:50.434378 5905 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:29:50.435976 5905 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:29:50.440287 6672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.438632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:37659 (local address 127.0.0.1:43448)
0809 16:29:50.439040 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:50.439049 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:50.439094 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:50.439247 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:50.439254 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:50.439261 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:50.439817 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:29:50.439819 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:50.439838 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:29:50.439865 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:50.440120 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:50.440126 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:50.440130 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:50.440178 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 16:29:50.440194 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":2,"spinlock_wait_cycles":896,"thread_start_us":103,"threads_started":1}
I20260809 16:29:50.440572 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.438642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:46791 (local address 127.0.0.1:56842)
0809 16:29:50.439075 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:50.439086 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:50.439140 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:50.439675 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:50.439676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:50.439679 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:50.440129 (+ 450us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:29:50.440130 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:50.440138 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:29:50.440152 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:50.440417 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:50.440418 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:50.440419 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:29:50.440464 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:29:50.440480 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":125,"threads_started":1}
I20260809 16:29:50.440665 6674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.438849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43448 (local address 127.30.23.129:37659)
0809 16:29:50.439043 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:50.439048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:50.439096 (+ 48us) server_negotiation.cc:407] Connection header received
0809 16:29:50.439151 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:50.439154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:50.439165 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:50.439236 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:50.439959 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:50.439967 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:50.439968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:50.439973 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:50.440011 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:50.440040 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:50.440527 (+ 487us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:50.440554 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 16:29:50.440572 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260809 16:29:50.440685 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.438631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:34401 (local address 127.0.0.1:49042)
0809 16:29:50.438811 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:50.438850 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:50.438944 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:50.439454 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:50.439455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:50.439765 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:50.440315 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:29:50.440315 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:50.440321 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:29:50.440332 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:50.440574 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:50.440575 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:50.440575 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 16:29:50.440604 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 16:29:50.440616 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":69,"threads_started":1}
I20260809 16:29:50.440832 6676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.439057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.30.23.130:46791)
0809 16:29:50.439222 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:50.439227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:50.439238 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:29:50.439267 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:50.439269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:50.439277 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:50.439354 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:50.440243 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:50.440252 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:50.440253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:50.440257 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:50.440301 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:50.440332 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:50.440390 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:50.440727 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 16:29:50.440745 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260809 16:29:50.441584 6675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.438859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49042 (local address 127.30.23.131:34401)
0809 16:29:50.439287 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:50.439290 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:50.439301 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:29:50.439327 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:50.439328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:50.439337 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:50.439895 (+ 558us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:50.440438 (+ 543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:50.440444 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:50.440444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:50.440450 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:50.440483 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:50.440503 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:50.441449 (+ 946us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:50.441475 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 16:29:50.441491 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":123,"threads_started":1}
I20260809 16:29:50.441622 6102 tablet_service.cc:1467] Processing CreateTablet for tablet ff9c4568b9e24fa2895fc4d0bd4fe08e (DEFAULT_TABLE table=test-table [id=0c2c47543cea4da6830f24c39d95ee25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:50.441629 6318 tablet_service.cc:1467] Processing CreateTablet for tablet ff9c4568b9e24fa2895fc4d0bd4fe08e (DEFAULT_TABLE table=test-table [id=0c2c47543cea4da6830f24c39d95ee25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:50.442050 6102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff9c4568b9e24fa2895fc4d0bd4fe08e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:50.442055 6318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff9c4568b9e24fa2895fc4d0bd4fe08e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:50.442236 6550 tablet_service.cc:1467] Processing CreateTablet for tablet ff9c4568b9e24fa2895fc4d0bd4fe08e (DEFAULT_TABLE table=test-table [id=0c2c47543cea4da6830f24c39d95ee25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:50.442601 6550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff9c4568b9e24fa2895fc4d0bd4fe08e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:50.444955 6680 tablet_bootstrap.cc:492] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7: Bootstrap starting.
I20260809 16:29:50.445520 6681 tablet_bootstrap.cc:492] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937: Bootstrap starting.
I20260809 16:29:50.445832 6680 tablet_bootstrap.cc:654] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:50.445989 6682 tablet_bootstrap.cc:492] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e: Bootstrap starting.
I20260809 16:29:50.446173 6681 tablet_bootstrap.cc:654] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:50.446384 6680 log.cc:824] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:50.446496 6681 log.cc:824] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:50.446830 6682 tablet_bootstrap.cc:654] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:50.447171 6682 log.cc:824] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:50.447238 6680 tablet_bootstrap.cc:492] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7: No bootstrap required, opened a new log
I20260809 16:29:50.447314 6680 ts_tablet_manager.cc:1397] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:29:50.447556 6681 tablet_bootstrap.cc:492] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937: No bootstrap required, opened a new log
I20260809 16:29:50.447626 6681 ts_tablet_manager.cc:1397] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:29:50.449944 6682 tablet_bootstrap.cc:492] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e: No bootstrap required, opened a new log
I20260809 16:29:50.450030 6682 ts_tablet_manager.cc:1397] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 16:29:50.450073 6680 raft_consensus.cc:348] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5b0be24c734bb294b5c083bfd4ea1e" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34401 } } peers { permanent_uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37659 } } peers { permanent_uuid: "a0453e43f6864efaa2ca0e687d7e7937" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46791 } }
I20260809 16:29:50.450253 6680 raft_consensus.cc:374] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:50.450295 6680 raft_consensus.cc:729] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff34d67c679b47c59ecdcf8d5e7e51f7, State: Initialized, Role: FOLLOWER
I20260809 16:29:50.450433 6680 consensus_queue.cc:260] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5b0be24c734bb294b5c083bfd4ea1e" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34401 } } peers { permanent_uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37659 } } peers { permanent_uuid: "a0453e43f6864efaa2ca0e687d7e7937" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46791 } }
I20260809 16:29:50.450413 6681 raft_consensus.cc:348] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5b0be24c734bb294b5c083bfd4ea1e" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34401 } } peers { permanent_uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37659 } } peers { permanent_uuid: "a0453e43f6864efaa2ca0e687d7e7937" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46791 } }
I20260809 16:29:50.450558 6681 raft_consensus.cc:374] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:50.450606 6681 raft_consensus.cc:729] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0453e43f6864efaa2ca0e687d7e7937, State: Initialized, Role: FOLLOWER
I20260809 16:29:50.450730 6680 ts_tablet_manager.cc:1428] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 16:29:50.450723 6681 consensus_queue.cc:260] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5b0be24c734bb294b5c083bfd4ea1e" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34401 } } peers { permanent_uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37659 } } peers { permanent_uuid: "a0453e43f6864efaa2ca0e687d7e7937" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46791 } }
I20260809 16:29:50.450870 6204 heartbeater.cc:503] Master 127.30.23.190:44865 was elected leader, sending a full tablet report...
I20260809 16:29:50.450968 6681 ts_tablet_manager.cc:1428] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:29:50.451140 6440 heartbeater.cc:503] Master 127.30.23.190:44865 was elected leader, sending a full tablet report...
I20260809 16:29:50.452561 6682 raft_consensus.cc:348] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5b0be24c734bb294b5c083bfd4ea1e" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34401 } } peers { permanent_uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37659 } } peers { permanent_uuid: "a0453e43f6864efaa2ca0e687d7e7937" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46791 } }
I20260809 16:29:50.452710 6682 raft_consensus.cc:374] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:50.452749 6682 raft_consensus.cc:729] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5b0be24c734bb294b5c083bfd4ea1e, State: Initialized, Role: FOLLOWER
I20260809 16:29:50.452867 6682 consensus_queue.cc:260] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5b0be24c734bb294b5c083bfd4ea1e" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34401 } } peers { permanent_uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37659 } } peers { permanent_uuid: "a0453e43f6864efaa2ca0e687d7e7937" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46791 } }
I20260809 16:29:50.453110 6682 ts_tablet_manager.cc:1428] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:29:50.453171 6655 heartbeater.cc:503] Master 127.30.23.190:44865 was elected leader, sending a full tablet report...
I20260809 16:29:50.550463 6690 raft_consensus.cc:482] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:29:50.550617 6690 raft_consensus.cc:504] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5b0be24c734bb294b5c083bfd4ea1e" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34401 } } peers { permanent_uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37659 } } peers { permanent_uuid: "a0453e43f6864efaa2ca0e687d7e7937" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46791 } }
I20260809 16:29:50.551100 6690 leader_election.cc:288] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f5b0be24c734bb294b5c083bfd4ea1e (127.30.23.131:34401), ff34d67c679b47c59ecdcf8d5e7e51f7 (127.30.23.129:37659)
I20260809 16:29:50.552642 6442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.551390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:34401 (local address 127.30.23.130:50349)
0809 16:29:50.551482 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:50.551497 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:50.551562 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:50.551743 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:50.551744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:50.551746 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:50.552263 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:29:50.552264 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:50.552271 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:29:50.552284 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:50.552488 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:50.552489 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:50.552490 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:29:50.552549 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:29:50.552571 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 16:29:50.552642 6675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.551539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:50349 (local address 127.30.23.131:34401)
0809 16:29:50.551606 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:50.551608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:50.551619 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:29:50.551650 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:50.551652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:50.551660 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:50.551718 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:50.552395 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:50.552396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:50.552397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:50.552402 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:50.552417 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:50.552421 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:50.552493 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:50.552555 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 16:29:50.552574 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:29:50.552850 6674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.551532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:48175 (local address 127.30.23.129:37659)
0809 16:29:50.551581 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:50.551583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:50.551725 (+ 142us) server_negotiation.cc:407] Connection header received
0809 16:29:50.551757 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:50.551757 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:50.551763 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:50.551828 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:50.552546 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:50.552548 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:50.552548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:50.552553 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:50.552578 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:50.552582 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:50.552649 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:50.552736 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 16:29:50.552754 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:29:50.553015 6693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.551390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:37659 (local address 127.30.23.130:48175)
0809 16:29:50.551628 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:50.551638 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:50.551702 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:50.551867 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:50.551867 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:50.551886 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:50.552431 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:29:50.552431 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:50.552437 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:29:50.552448 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:50.552647 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:50.552647 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:50.552648 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:29:50.552928 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 16:29:50.552942 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":70,"thread_start_us":95,"threads_started":1}
I20260809 16:29:50.553063 6585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff9c4568b9e24fa2895fc4d0bd4fe08e" candidate_uuid: "a0453e43f6864efaa2ca0e687d7e7937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5b0be24c734bb294b5c083bfd4ea1e" is_pre_election: true
I20260809 16:29:50.553239 6585 raft_consensus.cc:2454] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0453e43f6864efaa2ca0e687d7e7937 in term 0.
I20260809 16:29:50.553435 6135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff9c4568b9e24fa2895fc4d0bd4fe08e" candidate_uuid: "a0453e43f6864efaa2ca0e687d7e7937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7" is_pre_election: true
I20260809 16:29:50.553534 6235 leader_election.cc:302] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f5b0be24c734bb294b5c083bfd4ea1e, a0453e43f6864efaa2ca0e687d7e7937; no voters:
I20260809 16:29:50.553597 6135 raft_consensus.cc:2454] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0453e43f6864efaa2ca0e687d7e7937 in term 0.
I20260809 16:29:50.553686 6690 raft_consensus.cc:2781] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:29:50.553771 6690 raft_consensus.cc:482] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:29:50.553798 6690 raft_consensus.cc:3037] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:50.554637 6690 raft_consensus.cc:504] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5b0be24c734bb294b5c083bfd4ea1e" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34401 } } peers { permanent_uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37659 } } peers { permanent_uuid: "a0453e43f6864efaa2ca0e687d7e7937" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46791 } }
I20260809 16:29:50.554808 6690 leader_election.cc:288] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [CANDIDATE]: Term 1 election: Requested vote from peers 0f5b0be24c734bb294b5c083bfd4ea1e (127.30.23.131:34401), ff34d67c679b47c59ecdcf8d5e7e51f7 (127.30.23.129:37659)
I20260809 16:29:50.554989 6135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff9c4568b9e24fa2895fc4d0bd4fe08e" candidate_uuid: "a0453e43f6864efaa2ca0e687d7e7937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7"
I20260809 16:29:50.555078 6135 raft_consensus.cc:3037] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:50.555819 6135 raft_consensus.cc:2454] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0453e43f6864efaa2ca0e687d7e7937 in term 1.
I20260809 16:29:50.555922 6585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff9c4568b9e24fa2895fc4d0bd4fe08e" candidate_uuid: "a0453e43f6864efaa2ca0e687d7e7937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5b0be24c734bb294b5c083bfd4ea1e"
I20260809 16:29:50.556007 6238 leader_election.cc:302] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0453e43f6864efaa2ca0e687d7e7937, ff34d67c679b47c59ecdcf8d5e7e51f7; no voters:
I20260809 16:29:50.556036 6585 raft_consensus.cc:3037] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:50.556747 6585 raft_consensus.cc:2454] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0453e43f6864efaa2ca0e687d7e7937 in term 1.
I20260809 16:29:50.556838 6690 raft_consensus.cc:2781] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:50.557009 6690 raft_consensus.cc:686] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [term 1 LEADER]: Becoming Leader. State: Replica: a0453e43f6864efaa2ca0e687d7e7937, State: Running, Role: LEADER
I20260809 16:29:50.557124 6690 consensus_queue.cc:237] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5b0be24c734bb294b5c083bfd4ea1e" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34401 } } peers { permanent_uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37659 } } peers { permanent_uuid: "a0453e43f6864efaa2ca0e687d7e7937" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46791 } }
I20260809 16:29:50.558148 5904 catalog_manager.cc:5495] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0453e43f6864efaa2ca0e687d7e7937 (127.30.23.130). New cstate: current_term: 1 leader_uuid: "a0453e43f6864efaa2ca0e687d7e7937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5b0be24c734bb294b5c083bfd4ea1e" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0453e43f6864efaa2ca0e687d7e7937" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46791 } health_report { overall_health: HEALTHY } } }
W20260809 16:29:50.559243 6441 tablet.cc:2369] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:29:50.587735 6676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.586561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56854 (local address 127.30.23.130:46791)
0809 16:29:50.586652 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:50.586654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:50.586664 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:29:50.586701 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:50.586702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:50.586716 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:50.586774 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:50.587542 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:50.587544 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:50.587544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:50.587548 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:50.587565 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:50.587569 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:50.587628 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:50.587666 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 16:29:50.587678 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 16:29:50.589763 6585 raft_consensus.cc:1261] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e [term 1 FOLLOWER]: Refusing update from remote peer a0453e43f6864efaa2ca0e687d7e7937: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:50.590117 6135 raft_consensus.cc:1261] T ff9c4568b9e24fa2895fc4d0bd4fe08e P ff34d67c679b47c59ecdcf8d5e7e51f7 [term 1 FOLLOWER]: Refusing update from remote peer a0453e43f6864efaa2ca0e687d7e7937: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:50.590365 6690 consensus_queue.cc:1035] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f5b0be24c734bb294b5c083bfd4ea1e" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:50.590518 6695 consensus_queue.cc:1035] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff34d67c679b47c59ecdcf8d5e7e51f7" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:50.593160 6700 mvcc.cc:205] Tried to move back new op lower bound from 7316656089452437504 to 7316656089322905600. Current Snapshot: MvccSnapshot[applied={T|T < 7316656089452437504}]
I20260809 16:29:50.594347 6701 mvcc.cc:205] Tried to move back new op lower bound from 7316656089452437504 to 7316656089322905600. Current Snapshot: MvccSnapshot[applied={T|T < 7316656089452437504}]
I20260809 16:29:50.596021 5904 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:50.596143 5904 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:50.597061 5904 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e2dc8047b7a4dcbbbb1846c73b0918b: Sending DeleteTablet for 3 replicas of tablet ff9c4568b9e24fa2895fc4d0bd4fe08e
I20260809 16:29:50.597468 6550 tablet_service.cc:1514] Processing DeleteTablet for tablet ff9c4568b9e24fa2895fc4d0bd4fe08e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:50 UTC) from {username='slave'} at 127.0.0.1:49042
I20260809 16:29:50.597649 6318 tablet_service.cc:1514] Processing DeleteTablet for tablet ff9c4568b9e24fa2895fc4d0bd4fe08e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:50 UTC) from {username='slave'} at 127.0.0.1:56842
I20260809 16:29:50.597760 6714 tablet_replica.cc:330] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e: stopping tablet replica
I20260809 16:29:50.597805 6102 tablet_service.cc:1514] Processing DeleteTablet for tablet ff9c4568b9e24fa2895fc4d0bd4fe08e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:50 UTC) from {username='slave'} at 127.0.0.1:43448
I20260809 16:29:50.597885 6714 raft_consensus.cc:2229] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:50.597999 6714 raft_consensus.cc:2258] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:50.598062 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 6000
I20260809 16:29:50.601183 6715 tablet_replica.cc:330] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937: stopping tablet replica
I20260809 16:29:50.601455 6715 raft_consensus.cc:2229] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:29:50.601701 6715 raft_consensus.cc:2258] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:50.605383 6715 ts_tablet_manager.cc:1905] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:50.608011 6715 ts_tablet_manager.cc:1918] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:29:50.608196 6715 log.cc:1196] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/ff9c4568b9e24fa2895fc4d0bd4fe08e
I20260809 16:29:50.608681 6715 ts_tablet_manager.cc:1939] T ff9c4568b9e24fa2895fc4d0bd4fe08e P a0453e43f6864efaa2ca0e687d7e7937: Deleting consensus metadata
I20260809 16:29:50.611625 5882 catalog_manager.cc:4841] TS a0453e43f6864efaa2ca0e687d7e7937 (127.30.23.130:46791): tablet ff9c4568b9e24fa2895fc4d0bd4fe08e (table test-table [id=0c2c47543cea4da6830f24c39d95ee25]) successfully deleted
W20260809 16:29:50.620975 5887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:37659 (error 108)
I20260809 16:29:50.621287 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 6208
I20260809 16:29:50.621409 6714 ts_tablet_manager.cc:1905] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:50.621645 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.621494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:37659 (local address 127.0.0.1:43456)
0809 16:29:50.621557 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:50.621599 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:37659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 16:29:50.624537 5887 catalog_manager.cc:4584] TS ff34d67c679b47c59ecdcf8d5e7e51f7 (127.30.23.129:37659): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff9c4568b9e24fa2895fc4d0bd4fe08e: Network error: Client connection negotiation failed: client connection to 127.30.23.129:37659: connect: Connection refused (error 111)
I20260809 16:29:50.633561 6714 ts_tablet_manager.cc:1918] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:29:50.633693 6714 log.cc:1196] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/ff9c4568b9e24fa2895fc4d0bd4fe08e
I20260809 16:29:50.633998 6714 ts_tablet_manager.cc:1939] T ff9c4568b9e24fa2895fc4d0bd4fe08e P 0f5b0be24c734bb294b5c083bfd4ea1e: Deleting consensus metadata
I20260809 16:29:50.634536 5883 catalog_manager.cc:4841] TS 0f5b0be24c734bb294b5c083bfd4ea1e (127.30.23.131:34401): tablet ff9c4568b9e24fa2895fc4d0bd4fe08e (table test-table [id=0c2c47543cea4da6830f24c39d95ee25]) successfully deleted
I20260809 16:29:50.636332 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 6443
I20260809 16:29:50.648474 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 5858
2026-08-09T16:29:50Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (830 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 16:29:50.678629 30814 mini_oidc.cc:152] Starting JWKS server
I20260809 16:29:50.679765 30814 webserver.cc:460] Webserver started at https://127.0.0.1:46151/ using document root <none> and password file <none>
I20260809 16:29:50.679908 30814 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 16:29:50.680128 30814 webserver.cc:460] Webserver started at http://127.0.0.1:44775/ using document root <none> and password file <none>
2026-08-09T16:29:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:50Z Disabled control of system clock
I20260809 16:29:50.689718 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:41703
--webserver_interface=127.30.23.190
--webserver_port=0
--builtin_ntp_servers=127.30.23.148:37995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:41703
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46151/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/testchainca.cert with env {}
W20260809 16:29:50.777313 6729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:50.777601 6729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:50.777639 6729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:50.778824 6729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:50.778892 6729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:50.778918 6729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:50.778949 6729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:50.778975 6729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:29:50.779002 6729 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:29:50.779031 6729 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46151/jwks.json
I20260809 16:29:50.780604 6729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:41703
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:41703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46151/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:50.780900 6729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:50.781131 6729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:50.786291 6736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:50.786248 6735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:50.786248 6740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:50.786798 6729 server_base.cc:1034] running on GCE node
I20260809 16:29:50.787092 6729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:50.787555 6729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:50.788738 6729 hybrid_clock.cc:648] HybridClock initialized: now 1786292990788715 us; error 43 us; skew 500 ppm
I20260809 16:29:50.790221 6729 webserver.cc:460] Webserver started at http://127.30.23.190:41887/ using document root <none> and password file <none>
I20260809 16:29:50.790495 6729 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:50.790570 6729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:50.790786 6729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:50.792253 6729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "641427cdce6b4297a30672cec7d9263c"
format_stamp: "Formatted at 2026-08-09 16:29:50 on dist-test-slave-j2vl"
server_key: "d4c3ffe2755dd30ee60e3a848e649471"
server_key_iv: "3f994b773bf7031e03af431af5a0424e"
server_key_version: "encryptionkey@0"
I20260809 16:29:50.792666 6729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "641427cdce6b4297a30672cec7d9263c"
format_stamp: "Formatted at 2026-08-09 16:29:50 on dist-test-slave-j2vl"
server_key: "d4c3ffe2755dd30ee60e3a848e649471"
server_key_iv: "3f994b773bf7031e03af431af5a0424e"
server_key_version: "encryptionkey@0"
I20260809 16:29:50.794235 6729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:29:50.795138 6747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:50.795408 6729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:50.795531 6729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "641427cdce6b4297a30672cec7d9263c"
format_stamp: "Formatted at 2026-08-09 16:29:50 on dist-test-slave-j2vl"
server_key: "d4c3ffe2755dd30ee60e3a848e649471"
server_key_iv: "3f994b773bf7031e03af431af5a0424e"
server_key_version: "encryptionkey@0"
I20260809 16:29:50.795614 6729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:50.807574 6729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:50.807942 6729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:50.808112 6729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:50.812860 6729 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:41703
I20260809 16:29:50.812901 6827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:41703 every 8 connection(s)
I20260809 16:29:50.813305 6729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:50.814306 6828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:50.814939 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 6729
I20260809 16:29:50.815056 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:50.815249 30814 external_mini_cluster.cc:1428] Setting key fee9d5c85f77f924cc2410aea44ebe5b
I20260809 16:29:50.817431 6828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c: Bootstrap starting.
I20260809 16:29:50.818065 6828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:50.818356 6828 log.cc:824] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:50.819106 6828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c: No bootstrap required, opened a new log
I20260809 16:29:50.821171 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.815816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.30.23.190:41703)
0809 16:29:50.816024 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:50.816035 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:50.816063 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:29:50.816573 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:50.816580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:50.816643 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:50.816767 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:50.817041 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:50.818086 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:50.818929 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:50.819519 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:50.819661 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:50.820396 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:50.820400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:50.820400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:50.820610 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:50.820652 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:50.820666 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:50.820737 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:50.820832 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 16:29:50.820922 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260809 16:29:50.821710 6828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641427cdce6b4297a30672cec7d9263c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41703 } }
I20260809 16:29:50.821867 6828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:50.821904 6828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641427cdce6b4297a30672cec7d9263c, State: Initialized, Role: FOLLOWER
I20260809 16:29:50.822026 6828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641427cdce6b4297a30672cec7d9263c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41703 } }
I20260809 16:29:50.822108 6828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:50.822161 6828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:50.822211 6828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:50.822276 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:29:50.822899 6828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641427cdce6b4297a30672cec7d9263c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41703 } }
I20260809 16:29:50.823014 6828 leader_election.cc:302] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 641427cdce6b4297a30672cec7d9263c; no voters:
I20260809 16:29:50.823179 6828 leader_election.cc:288] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:50.823249 6837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:50.823431 6837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [term 1 LEADER]: Becoming Leader. State: Replica: 641427cdce6b4297a30672cec7d9263c, State: Running, Role: LEADER
I20260809 16:29:50.823478 6828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:50.823525 6837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641427cdce6b4297a30672cec7d9263c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41703 } }
I20260809 16:29:50.823938 6840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 641427cdce6b4297a30672cec7d9263c. Latest consensus state: current_term: 1 leader_uuid: "641427cdce6b4297a30672cec7d9263c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641427cdce6b4297a30672cec7d9263c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41703 } } }
I20260809 16:29:50.823973 6838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "641427cdce6b4297a30672cec7d9263c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641427cdce6b4297a30672cec7d9263c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41703 } } }
I20260809 16:29:50.824047 6840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:50.824054 6838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:50.824436 6847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:50.825013 6847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:50.826772 6847 catalog_manager.cc:1347] Generated new cluster ID: 7ac077be9c0449bbaffb0d275cd9135e
I20260809 16:29:50.826844 6847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:29:50.831416 6847 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:50.831952 6847 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:50.841315 6847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 641427cdce6b4297a30672cec7d9263c: Generated new TSK 0
I20260809 16:29:50.841600 6847 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 16:29:50.870854 6876 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 16:29:50.870988 6876 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 16:29:50.872400 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:50.854775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40852 (local address 127.30.23.190:41703)
0809 16:29:50.854842 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:50.854844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:50.854987 (+ 143us) server_negotiation.cc:407] Connection header received
0809 16:29:50.855072 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:50.855074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:50.855088 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:50.855167 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:29:50.855308 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:50.856126 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:50.856968 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:50.857288 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:50.857390 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:50.857640 (+ 250us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:29:50.872081 (+ 14441us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 16:29:50.872297 (+ 216us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40852: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 16:29:50.872493 6834 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40852: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 16:29:50.872910 6875 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:29:50.854646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:41703 (local address 127.0.0.1:40852)
0809 16:29:50.854907 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:50.854919 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:50.854985 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:50.855181 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:50.855182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:50.855184 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:29:50.855243 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:50.855245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:50.856235 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:50.856237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:50.856871 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:50.856873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:50.857435 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:50.857436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:50.857489 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:50.857512 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:29:50.872295 (+ 14783us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 16:29:50.872421 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.23.190:41703: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":110,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.23.190:41703: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46151/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 16:29:50.873495 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 6729
2026-08-09T16:29:50Z chronyd exiting
I20260809 16:29:50.921501 30814 test_util.cc:175] -----------------------------------------------
I20260809 16:29:50.921593 30814 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786292963539133-30814-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (244 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 16:29:50.922430 30814 mini_oidc.cc:152] Starting JWKS server
I20260809 16:29:50.923184 30814 webserver.cc:460] Webserver started at https://127.0.0.1:45299/ using document root <none> and password file <none>
I20260809 16:29:50.923257 30814 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 16:29:50.923414 30814 webserver.cc:460] Webserver started at http://127.0.0.1:38555/ using document root <none> and password file <none>
2026-08-09T16:29:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:50Z Disabled control of system clock
I20260809 16:29:50.932893 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:45055
--webserver_interface=127.30.23.190
--webserver_port=0
--builtin_ntp_servers=127.30.23.148:45887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:45055
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45299/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/testchainca.cert with env {}
W20260809 16:29:51.016654 6891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:51.016914 6891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:51.016959 6891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:51.018179 6891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:51.018245 6891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:51.018267 6891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:51.018290 6891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:51.018330 6891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:29:51.018358 6891 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:29:51.018411 6891 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45299/jwks.json
I20260809 16:29:51.019994 6891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:45055
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:45055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45299/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:51.020280 6891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:51.020514 6891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:51.025732 6899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:51.025805 6897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:51.025882 6903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:51.025928 6891 server_base.cc:1034] running on GCE node
I20260809 16:29:51.026427 6891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:51.026804 6891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:51.028002 6891 hybrid_clock.cc:648] HybridClock initialized: now 1786292991027987 us; error 32 us; skew 500 ppm
I20260809 16:29:51.029541 6891 webserver.cc:460] Webserver started at http://127.30.23.190:41179/ using document root <none> and password file <none>
I20260809 16:29:51.029808 6891 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:51.029858 6891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:51.030001 6891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:51.031080 6891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "6a88ce9ce4d84ed0b02ab978c3bc25a5"
format_stamp: "Formatted at 2026-08-09 16:29:51 on dist-test-slave-j2vl"
server_key: "2ce726b6861af4ae36a5f62120ba8d12"
server_key_iv: "56a8f3303abd4f0ccd476db60492c544"
server_key_version: "encryptionkey@0"
I20260809 16:29:51.031446 6891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "6a88ce9ce4d84ed0b02ab978c3bc25a5"
format_stamp: "Formatted at 2026-08-09 16:29:51 on dist-test-slave-j2vl"
server_key: "2ce726b6861af4ae36a5f62120ba8d12"
server_key_iv: "56a8f3303abd4f0ccd476db60492c544"
server_key_version: "encryptionkey@0"
I20260809 16:29:51.032820 6891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:29:51.033699 6913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:51.033955 6891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:51.034072 6891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "6a88ce9ce4d84ed0b02ab978c3bc25a5"
format_stamp: "Formatted at 2026-08-09 16:29:51 on dist-test-slave-j2vl"
server_key: "2ce726b6861af4ae36a5f62120ba8d12"
server_key_iv: "56a8f3303abd4f0ccd476db60492c544"
server_key_version: "encryptionkey@0"
I20260809 16:29:51.034150 6891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:51.043912 6891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:51.044247 6891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:51.044410 6891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:51.048764 6891 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:45055
I20260809 16:29:51.048803 6992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:45055 every 8 connection(s)
I20260809 16:29:51.049181 6891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:51.049996 6994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:51.052731 6994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5: Bootstrap starting.
I20260809 16:29:51.053292 6994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:51.053617 6994 log.cc:824] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:51.054302 6994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5: No bootstrap required, opened a new log
I20260809 16:29:51.055876 6994 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a88ce9ce4d84ed0b02ab978c3bc25a5" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45055 } }
I20260809 16:29:51.056021 6994 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:51.056046 6994 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a88ce9ce4d84ed0b02ab978c3bc25a5, State: Initialized, Role: FOLLOWER
I20260809 16:29:51.056146 6994 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a88ce9ce4d84ed0b02ab978c3bc25a5" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45055 } }
I20260809 16:29:51.056200 6994 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:51.056219 6994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:51.056288 6994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:51.056839 6994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a88ce9ce4d84ed0b02ab978c3bc25a5" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45055 } }
I20260809 16:29:51.056962 6994 leader_election.cc:302] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a88ce9ce4d84ed0b02ab978c3bc25a5; no voters:
I20260809 16:29:51.057140 6994 leader_election.cc:288] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:51.057215 7000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:51.057416 6994 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:51.057420 7000 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [term 1 LEADER]: Becoming Leader. State: Replica: 6a88ce9ce4d84ed0b02ab978c3bc25a5, State: Running, Role: LEADER
I20260809 16:29:51.057574 7000 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a88ce9ce4d84ed0b02ab978c3bc25a5" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45055 } }
I20260809 16:29:51.057940 7003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a88ce9ce4d84ed0b02ab978c3bc25a5. Latest consensus state: current_term: 1 leader_uuid: "6a88ce9ce4d84ed0b02ab978c3bc25a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a88ce9ce4d84ed0b02ab978c3bc25a5" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45055 } } }
I20260809 16:29:51.058025 7003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:51.058007 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 6891
I20260809 16:29:51.058125 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:51.058099 7002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a88ce9ce4d84ed0b02ab978c3bc25a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a88ce9ce4d84ed0b02ab978c3bc25a5" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45055 } } }
I20260809 16:29:51.058157 7002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:51.058269 7014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:51.058336 30814 external_mini_cluster.cc:1428] Setting key 06cd0c9cac30de841c8fdc0b0a90a738
I20260809 16:29:51.058779 7014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:51.060367 7014 catalog_manager.cc:1347] Generated new cluster ID: 20cdb8621f534e38a8257d91c837a1aa
I20260809 16:29:51.060441 7014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:29:51.064723 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:51.059008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36520 (local address 127.30.23.190:45055)
0809 16:29:51.059290 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:51.059299 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:51.059334 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:29:51.059574 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:51.059583 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:51.059670 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:51.059823 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:51.060078 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.061146 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.062255 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.062868 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.063163 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:51.063868 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:51.063872 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:51.063872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:51.064098 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:51.064149 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:51.064165 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:51.064242 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:51.064318 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 16:29:51.064508 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260809 16:29:51.065618 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:29:51.077824 7014 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:51.078373 7014 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:51.081877 7014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a88ce9ce4d84ed0b02ab978c3bc25a5: Generated new TSK 0
I20260809 16:29:51.082068 7014 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 16:29:51.102141 7047 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 16:29:51.102244 7047 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 16:29:51.102655 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:51.087698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36522 (local address 127.30.23.190:45055)
0809 16:29:51.087742 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:51.087744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:51.087916 (+ 172us) server_negotiation.cc:407] Connection header received
0809 16:29:51.088060 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:51.088062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:51.088078 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:51.088160 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:29:51.088355 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.089259 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.090298 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.090530 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.090617 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:51.090805 (+ 188us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:29:51.102383 (+ 11578us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45299/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 16:29:51.102565 (+ 182us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36522: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45299/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 16:29:51.102723 7031 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36522: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45299/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 16:29:51.102725 7046 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:29:51.087605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:45055 (local address 127.0.0.1:36522)
0809 16:29:51.087801 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:51.087870 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:51.087935 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:51.088212 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:51.088213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:51.088215 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:29:51.088276 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:51.088279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.089412 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.089413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:51.090205 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:51.090208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.090623 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.090625 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:51.090688 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:51.090700 (+ 12us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:29:51.102510 (+ 11810us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45299/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 16:29:51.102621 (+ 111us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.23.190:45055: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45299/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":80,"threads_started":1}
I20260809 16:29:51.103124 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 6891
2026-08-09T16:29:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (227 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 16:29:51.149960 30814 mini_oidc.cc:152] Starting JWKS server
I20260809 16:29:51.150645 30814 webserver.cc:460] Webserver started at https://127.0.0.1:37223/ using document root <none> and password file <none>
I20260809 16:29:51.150720 30814 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 16:29:51.150878 30814 webserver.cc:460] Webserver started at http://127.0.0.1:45859/ using document root <none> and password file <none>
2026-08-09T16:29:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:51Z Disabled control of system clock
I20260809 16:29:51.160543 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:39557
--webserver_interface=127.30.23.190
--webserver_port=0
--builtin_ntp_servers=127.30.23.148:45265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:39557
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37223/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 16:29:51.246459 7056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:51.246711 7056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:51.246737 7056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:51.247953 7056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:51.247999 7056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:51.248013 7056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:51.248026 7056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:51.248039 7056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:29:51.248051 7056 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:29:51.248063 7056 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37223/jwks.json
I20260809 16:29:51.249496 7056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:39557
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:39557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37223/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:51.249727 7056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:51.250001 7056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:51.255560 7068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:51.255566 7062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:51.255566 7065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:51.255946 7056 server_base.cc:1034] running on GCE node
I20260809 16:29:51.256238 7056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:51.256568 7056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:51.257776 7056 hybrid_clock.cc:648] HybridClock initialized: now 1786292991257748 us; error 63 us; skew 500 ppm
I20260809 16:29:51.259580 7056 webserver.cc:460] Webserver started at http://127.30.23.190:42829/ using document root <none> and password file <none>
I20260809 16:29:51.259923 7056 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:51.260015 7056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:51.260195 7056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:51.261554 7056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "718632dadae848f093ef953cbae6b303"
format_stamp: "Formatted at 2026-08-09 16:29:51 on dist-test-slave-j2vl"
server_key: "1234a36c628c5455bad831b82876a0e4"
server_key_iv: "c7b51aa7a551cab9db454e2041367a5b"
server_key_version: "encryptionkey@0"
I20260809 16:29:51.261974 7056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "718632dadae848f093ef953cbae6b303"
format_stamp: "Formatted at 2026-08-09 16:29:51 on dist-test-slave-j2vl"
server_key: "1234a36c628c5455bad831b82876a0e4"
server_key_iv: "c7b51aa7a551cab9db454e2041367a5b"
server_key_version: "encryptionkey@0"
I20260809 16:29:51.263542 7056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:29:51.264518 7075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:51.264739 7056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:51.264855 7056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "718632dadae848f093ef953cbae6b303"
format_stamp: "Formatted at 2026-08-09 16:29:51 on dist-test-slave-j2vl"
server_key: "1234a36c628c5455bad831b82876a0e4"
server_key_iv: "c7b51aa7a551cab9db454e2041367a5b"
server_key_version: "encryptionkey@0"
I20260809 16:29:51.264946 7056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:51.296453 7056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:51.296850 7056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:51.297027 7056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:51.301815 7056 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:39557
I20260809 16:29:51.301855 7166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:39557 every 8 connection(s)
I20260809 16:29:51.302264 7056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:51.303063 7167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:51.305997 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 7056
I20260809 16:29:51.306059 7167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303: Bootstrap starting.
I20260809 16:29:51.306119 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:51.306321 30814 external_mini_cluster.cc:1428] Setting key 381e894648a67e7f90f21b92025c8ace
I20260809 16:29:51.306840 7167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:51.307204 7167 log.cc:824] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:51.308163 7167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303: No bootstrap required, opened a new log
I20260809 16:29:51.310504 7167 raft_consensus.cc:348] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718632dadae848f093ef953cbae6b303" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39557 } }
I20260809 16:29:51.310732 7167 raft_consensus.cc:374] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:51.310789 7167 raft_consensus.cc:729] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 718632dadae848f093ef953cbae6b303, State: Initialized, Role: FOLLOWER
I20260809 16:29:51.310959 7167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718632dadae848f093ef953cbae6b303" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39557 } }
I20260809 16:29:51.311060 7167 raft_consensus.cc:388] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:51.311105 7167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:51.311156 7167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:51.312026 7167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718632dadae848f093ef953cbae6b303" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39557 } }
I20260809 16:29:51.312142 7167 leader_election.cc:302] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 718632dadae848f093ef953cbae6b303; no voters:
I20260809 16:29:51.312340 7167 leader_election.cc:288] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:51.312391 7174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:51.312561 7174 raft_consensus.cc:686] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [term 1 LEADER]: Becoming Leader. State: Replica: 718632dadae848f093ef953cbae6b303, State: Running, Role: LEADER
I20260809 16:29:51.312651 7174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718632dadae848f093ef953cbae6b303" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39557 } }
I20260809 16:29:51.312649 7167 sys_catalog.cc:564] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:51.313000 7172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:51.306902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50154 (local address 127.30.23.190:39557)
0809 16:29:51.307134 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:51.307142 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:51.307169 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:29:51.307889 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:51.307898 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:51.307973 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:51.308097 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:51.308422 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.309484 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.310651 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.311250 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.311370 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:51.312056 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:51.312059 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:51.312059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:51.312306 (+ 247us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:51.312381 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:51.312397 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:51.312495 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:51.312684 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 16:29:51.312782 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260809 16:29:51.313032 7177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 718632dadae848f093ef953cbae6b303. Latest consensus state: current_term: 1 leader_uuid: "718632dadae848f093ef953cbae6b303" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718632dadae848f093ef953cbae6b303" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39557 } } }
I20260809 16:29:51.313122 7177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:51.313036 7176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "718632dadae848f093ef953cbae6b303" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "718632dadae848f093ef953cbae6b303" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39557 } } }
I20260809 16:29:51.313323 7176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:51.313326 7181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:51.314436 7181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:51.314486 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:29:51.315976 7181 catalog_manager.cc:1347] Generated new cluster ID: 0c25fc684cb34266b2c36fb47f7553ad
I20260809 16:29:51.316044 7181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:29:51.328394 7181 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:51.328984 7181 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:51.336913 7181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 718632dadae848f093ef953cbae6b303: Generated new TSK 0
I20260809 16:29:51.337136 7181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:29:51.349731 7172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:51.345397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50156 (local address 127.30.23.190:39557)
0809 16:29:51.345444 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:51.345446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:51.345503 (+ 57us) server_negotiation.cc:407] Connection header received
0809 16:29:51.345596 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:51.345597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:51.345608 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:51.345672 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:29:51.345838 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.346362 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.347545 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.347751 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.347808 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:51.348059 (+ 251us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:29:51.349465 (+ 1406us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37223/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 16:29:51.349639 (+ 174us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50156: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37223/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 16:29:51.349818 7172 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50156: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37223/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 16:29:51.352703 7208 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:29:51.345239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:39557 (local address 127.0.0.1:50156)
0809 16:29:51.345438 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:51.345450 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:51.345524 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:51.345695 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:51.345696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:51.345698 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:29:51.345764 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:51.345767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.346452 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.346453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:51.347445 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:51.347448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.347844 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.347845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:51.347912 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:51.347919 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:29:51.349594 (+ 1675us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37223/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 16:29:51.352616 (+ 3022us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.23.190:39557: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37223/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":2793,"thread_start_us":152,"threads_started":1}
I20260809 16:29:51.353251 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 7056
E20260809 16:29:51.369451 7209 webserver.cc:533] Webserver: sslize failed:
E20260809 16:29:51.369540 7209 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T16:29:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (249 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T16:29:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:51Z Disabled control of system clock
I20260809 16:29:51.408820 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:42967
--webserver_interface=127.30.23.190
--webserver_port=0
--builtin_ntp_servers=127.30.23.148:43847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:42967
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/testchainca.cert with env {}
W20260809 16:29:51.493712 7219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:51.493960 7219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:51.493986 7219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:51.495311 7219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:51.495359 7219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:51.495373 7219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:51.495385 7219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:51.495397 7219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:29:51.495437 7219 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:29:51.495467 7219 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 16:29:51.497104 7219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:42967
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:42967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:51.497426 7219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:51.497666 7219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:51.502924 7229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:51.502938 7228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:51.503011 7219 server_base.cc:1034] running on GCE node
W20260809 16:29:51.502944 7231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:51.503362 7219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:51.503695 7219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:51.504873 7219 hybrid_clock.cc:648] HybridClock initialized: now 1786292991504872 us; error 50 us; skew 500 ppm
I20260809 16:29:51.506274 7219 webserver.cc:460] Webserver started at http://127.30.23.190:39975/ using document root <none> and password file <none>
I20260809 16:29:51.506559 7219 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:51.506611 7219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:51.506785 7219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:51.507910 7219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "2d9862cc2d0043e4b3bee1dcad6ccd81"
format_stamp: "Formatted at 2026-08-09 16:29:51 on dist-test-slave-j2vl"
server_key: "47a4a64b71d0a78e41514752f0ce410d"
server_key_iv: "1ee866d0baee9a0426b094023e569a8e"
server_key_version: "encryptionkey@0"
I20260809 16:29:51.508299 7219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "2d9862cc2d0043e4b3bee1dcad6ccd81"
format_stamp: "Formatted at 2026-08-09 16:29:51 on dist-test-slave-j2vl"
server_key: "47a4a64b71d0a78e41514752f0ce410d"
server_key_iv: "1ee866d0baee9a0426b094023e569a8e"
server_key_version: "encryptionkey@0"
I20260809 16:29:51.509861 7219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:29:51.510784 7241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:51.510967 7219 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:51.511067 7219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "2d9862cc2d0043e4b3bee1dcad6ccd81"
format_stamp: "Formatted at 2026-08-09 16:29:51 on dist-test-slave-j2vl"
server_key: "47a4a64b71d0a78e41514752f0ce410d"
server_key_iv: "1ee866d0baee9a0426b094023e569a8e"
server_key_version: "encryptionkey@0"
I20260809 16:29:51.511128 7219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:51.529829 7219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:51.530169 7219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:51.530284 7219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:51.534842 7219 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:42967
I20260809 16:29:51.534904 7343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:42967 every 8 connection(s)
I20260809 16:29:51.535259 7219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:51.536415 7347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:51.538941 7347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81: Bootstrap starting.
I20260809 16:29:51.539549 7347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:51.539824 7347 log.cc:824] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:51.540603 7347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81: No bootstrap required, opened a new log
I20260809 16:29:51.542457 7347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9862cc2d0043e4b3bee1dcad6ccd81" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 42967 } }
I20260809 16:29:51.542614 7347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:51.542641 7347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d9862cc2d0043e4b3bee1dcad6ccd81, State: Initialized, Role: FOLLOWER
I20260809 16:29:51.542770 7347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9862cc2d0043e4b3bee1dcad6ccd81" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 42967 } }
I20260809 16:29:51.542863 7347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:51.542903 7347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:51.542986 7347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:51.543610 7347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9862cc2d0043e4b3bee1dcad6ccd81" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 42967 } }
I20260809 16:29:51.543762 7347 leader_election.cc:302] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d9862cc2d0043e4b3bee1dcad6ccd81; no voters:
I20260809 16:29:51.543968 7347 leader_election.cc:288] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:51.544178 7351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:51.544343 7351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [term 1 LEADER]: Becoming Leader. State: Replica: 2d9862cc2d0043e4b3bee1dcad6ccd81, State: Running, Role: LEADER
I20260809 16:29:51.544402 7347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:51.544386 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 7219
I20260809 16:29:51.544438 7351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9862cc2d0043e4b3bee1dcad6ccd81" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 42967 } }
I20260809 16:29:51.544492 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:51.544703 30814 external_mini_cluster.cc:1428] Setting key 6d8e8c615bfa8da46b7b6d78dae46b27
I20260809 16:29:51.544744 7352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d9862cc2d0043e4b3bee1dcad6ccd81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9862cc2d0043e4b3bee1dcad6ccd81" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 42967 } } }
I20260809 16:29:51.544831 7352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:51.545025 7353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d9862cc2d0043e4b3bee1dcad6ccd81. Latest consensus state: current_term: 1 leader_uuid: "2d9862cc2d0043e4b3bee1dcad6ccd81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9862cc2d0043e4b3bee1dcad6ccd81" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 42967 } } }
I20260809 16:29:51.545100 7356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:51.545147 7353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:51.545764 7356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:51.547580 7356 catalog_manager.cc:1347] Generated new cluster ID: 98436a6e53b1492d89d43914d53d1ae4
I20260809 16:29:51.547648 7356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:29:51.551831 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:51.545450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60466 (local address 127.30.23.190:42967)
0809 16:29:51.546006 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:51.546015 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:51.546047 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:29:51.546162 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:51.546170 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:51.546245 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:51.546787 (+ 542us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:51.546882 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.548281 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.549550 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.550159 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.550275 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:51.551078 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:51.551082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:51.551083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:29:51.551305 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:51.551346 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:29:51.551361 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:51.551428 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:51.551540 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 16:29:51.551618 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":73,"threads_started":1}
I20260809 16:29:51.552762 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:29:51.562713 7356 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:51.563568 7356 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:51.570060 7356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d9862cc2d0043e4b3bee1dcad6ccd81: Generated new TSK 0
I20260809 16:29:51.570277 7356 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:29:51.656351 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:51.580185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60468 (local address 127.30.23.190:42967)
0809 16:29:51.580256 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:51.580258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:51.580378 (+ 120us) server_negotiation.cc:407] Connection header received
0809 16:29:51.580462 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:51.580463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:51.580477 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:51.580555 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:29:51.580747 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.581590 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.582497 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.582858 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.582932 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:51.583149 (+ 217us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:29:51.656041 (+ 72892us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 16:29:51.656271 (+ 230us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60468: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 16:29:51.656451 7366 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60468: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 16:29:51.656509 7386 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:29:51.580090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:42967 (local address 127.0.0.1:60468)
0809 16:29:51.580283 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:51.580296 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:51.580374 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:51.580594 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:51.580596 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:51.580598 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:29:51.580656 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:51.580658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.581689 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.581690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:51.582405 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:51.582408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:51.582970 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:51.582971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:51.583024 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:51.583030 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:29:51.656233 (+ 73203us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 16:29:51.656371 (+ 138us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.23.190:42967: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":151,"threads_started":1}
I20260809 16:29:51.656790 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 7219
2026-08-09T16:29:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (285 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (114 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (96 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:29:51 dist-test-slave-j2vl krb5kdc[7409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:29:51 dist-test-slave-j2vl krb5kdc[7409](info): set up 2 sockets
Aug 09 16:29:51 dist-test-slave-j2vl krb5kdc[7409](info): commencing operation
krb5kdc: starting...
W20260809 16:29:53.168653 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:29:53 dist-test-slave-j2vl krb5kdc[7409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:29:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:29:53.296708 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:36621
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:36621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:53.379436 7437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:53.379688 7437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:53.379727 7437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:53.380920 7437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:53.380999 7437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:53.381023 7437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:53.381044 7437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:53.381072 7437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:53.382884 7437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:36621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:36621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:53.383203 7437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:53.383457 7437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:53.388535 7452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:53.388540 7449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:53.388548 7448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:53.389169 7437 server_base.cc:1034] running on GCE node
I20260809 16:29:53.389489 7437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:53.389914 7437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:53.391093 7437 hybrid_clock.cc:648] HybridClock initialized: now 1786292993391059 us; error 50 us; skew 500 ppm
Aug 09 16:29:53 dist-test-slave-j2vl krb5kdc[7409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292993, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:53.394142 7437 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:29:53.394678 7437 webserver.cc:460] Webserver started at http://127.30.23.190:40713/ using document root <none> and password file <none>
I20260809 16:29:53.394948 7437 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:53.394995 7437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:53.395154 7437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:53.396338 7437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "33b611e7ab3d42e8a74ee72844965fc9"
format_stamp: "Formatted at 2026-08-09 16:29:53 on dist-test-slave-j2vl"
server_key: "258ee6ef2e1cbc60b79e3936eea28ea5"
server_key_iv: "b5606f2c0786b286ea60fe49f6f19fe8"
server_key_version: "encryptionkey@0"
I20260809 16:29:53.396818 7437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "33b611e7ab3d42e8a74ee72844965fc9"
format_stamp: "Formatted at 2026-08-09 16:29:53 on dist-test-slave-j2vl"
server_key: "258ee6ef2e1cbc60b79e3936eea28ea5"
server_key_iv: "b5606f2c0786b286ea60fe49f6f19fe8"
server_key_version: "encryptionkey@0"
I20260809 16:29:53.398386 7437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:29:53.399204 7461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:53.399400 7437 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:53.399520 7437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "33b611e7ab3d42e8a74ee72844965fc9"
format_stamp: "Formatted at 2026-08-09 16:29:53 on dist-test-slave-j2vl"
server_key: "258ee6ef2e1cbc60b79e3936eea28ea5"
server_key_iv: "b5606f2c0786b286ea60fe49f6f19fe8"
server_key_version: "encryptionkey@0"
I20260809 16:29:53.399596 7437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:53.437791 7437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:53.440140 7437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:53.440335 7437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:53.444773 7437 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:36621
I20260809 16:29:53.444809 7549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:36621 every 8 connection(s)
I20260809 16:29:53.445190 7437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:53.446190 7551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:53.448645 7551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9: Bootstrap starting.
I20260809 16:29:53.449240 7551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:53.449605 7551 log.cc:824] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:53.450346 7551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9: No bootstrap required, opened a new log
I20260809 16:29:53.451992 7551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b611e7ab3d42e8a74ee72844965fc9" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36621 } }
I20260809 16:29:53.452145 7551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:53.452168 7551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b611e7ab3d42e8a74ee72844965fc9, State: Initialized, Role: FOLLOWER
I20260809 16:29:53.452320 7551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b611e7ab3d42e8a74ee72844965fc9" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36621 } }
I20260809 16:29:53.452407 7551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:53.452431 7551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:53.452471 7551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:53.453063 7551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b611e7ab3d42e8a74ee72844965fc9" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36621 } }
I20260809 16:29:53.453182 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 7437
I20260809 16:29:53.453197 7551 leader_election.cc:302] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33b611e7ab3d42e8a74ee72844965fc9; no voters:
I20260809 16:29:53.453282 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:53.453424 7551 leader_election.cc:288] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:53.453536 30814 external_mini_cluster.cc:1428] Setting key 0fa4ccc50436964a9db4131cc488a48f
I20260809 16:29:53.453548 7556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:53.453804 7551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:53.454039 7556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [term 1 LEADER]: Becoming Leader. State: Replica: 33b611e7ab3d42e8a74ee72844965fc9, State: Running, Role: LEADER
I20260809 16:29:53.454407 7556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b611e7ab3d42e8a74ee72844965fc9" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36621 } }
I20260809 16:29:53.455016 7558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33b611e7ab3d42e8a74ee72844965fc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b611e7ab3d42e8a74ee72844965fc9" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36621 } } }
I20260809 16:29:53.455113 7558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:53.455499 7559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33b611e7ab3d42e8a74ee72844965fc9. Latest consensus state: current_term: 1 leader_uuid: "33b611e7ab3d42e8a74ee72844965fc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b611e7ab3d42e8a74ee72844965fc9" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36621 } } }
I20260809 16:29:53.455650 7576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:53.455783 7559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:53.456224 7576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:53.457703 7576 catalog_manager.cc:1347] Generated new cluster ID: 687bfe2d8b844460b3bd0e30cd815f7f
I20260809 16:29:53.457774 7576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:29:53 dist-test-slave-j2vl krb5kdc[7409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:53.465039 7576 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:53.466033 7576 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:53.468938 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:53.454190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40852 (local address 127.30.23.190:36621)
0809 16:29:53.454370 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:53.454383 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:53.454434 (+ 51us) server_negotiation.cc:407] Connection header received
0809 16:29:53.454582 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:53.454590 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:53.454663 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:53.455517 (+ 854us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:53.455767 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:53.457163 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:53.458282 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:53.458907 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:53.459028 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:53.463030 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:53.463037 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:53.463037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:53.463082 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:53.466449 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:53.467001 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:53.467005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:53.467005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:53.467135 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:53.467389 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:53.467390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:53.467390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:53.467764 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:53.467964 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:53.468502 (+ 538us) server_negotiation.cc:299] Negotiation successful
0809 16:29:53.468686 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260809 16:29:53.477725 7576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33b611e7ab3d42e8a74ee72844965fc9: Generated new TSK 0
I20260809 16:29:53.477962 7576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:29:53.521509 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:36621
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:53.613423 7587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:53.613699 7587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:53.613744 7587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:53.615185 7587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:53.615257 7587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:53.615305 7587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:29:53.617051 7587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:36621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:53.617329 7587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:53.617599 7587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:29:53.623175 7587 server_base.cc:1034] running on GCE node
W20260809 16:29:53.623142 7599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:53.623162 7602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:53.623142 7604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:53.623575 7587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:53.623852 7587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:53.625041 7587 hybrid_clock.cc:648] HybridClock initialized: now 1786292993624987 us; error 87 us; skew 500 ppm
Aug 09 16:29:53 dist-test-slave-j2vl krb5kdc[7409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292993, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:53.628243 7587 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:29:53.628746 7587 webserver.cc:460] Webserver started at http://127.30.23.129:41815/ using document root <none> and password file <none>
I20260809 16:29:53.628960 7587 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:53.629004 7587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:53.629098 7587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:53.630211 7587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "fbcfd81eb1404cc89610bc0c64b3d52e"
format_stamp: "Formatted at 2026-08-09 16:29:53 on dist-test-slave-j2vl"
server_key: "23b10744610828bc2791e4eff25b2ca8"
server_key_iv: "6031686fd28c14d9db1d081edfe20d05"
server_key_version: "encryptionkey@0"
I20260809 16:29:53.630530 7587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "fbcfd81eb1404cc89610bc0c64b3d52e"
format_stamp: "Formatted at 2026-08-09 16:29:53 on dist-test-slave-j2vl"
server_key: "23b10744610828bc2791e4eff25b2ca8"
server_key_iv: "6031686fd28c14d9db1d081edfe20d05"
server_key_version: "encryptionkey@0"
I20260809 16:29:53.632149 7587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:29:53.633038 7618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:53.633229 7587 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:53.633328 7587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "fbcfd81eb1404cc89610bc0c64b3d52e"
format_stamp: "Formatted at 2026-08-09 16:29:53 on dist-test-slave-j2vl"
server_key: "23b10744610828bc2791e4eff25b2ca8"
server_key_iv: "6031686fd28c14d9db1d081edfe20d05"
server_key_version: "encryptionkey@0"
I20260809 16:29:53.633451 7587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:53.646994 7587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:53.649526 7587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:53.649829 7587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:53.650192 7587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:53.650547 7587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:53.650593 7587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:53.650619 7587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:53.650631 7587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:53.657111 7587 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:39503
I20260809 16:29:53.657187 7818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:39503 every 8 connection(s)
I20260809 16:29:53.657588 7587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:29:53.657925 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 7587
I20260809 16:29:53.658041 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:29:53.658298 30814 external_mini_cluster.cc:1428] Setting key 099b2d6e4b2202960dbbcec5d8710682
Aug 09 16:29:53 dist-test-slave-j2vl krb5kdc[7409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292993, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
I20260809 16:29:53.671195 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:53.658652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:37085 (local address 127.30.23.190:36621)
0809 16:29:53.658736 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:53.658739 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:53.659128 (+ 389us) server_negotiation.cc:407] Connection header received
0809 16:29:53.660761 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:53.660762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:53.660773 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:53.660833 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:53.661732 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:53.662720 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:53.663821 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:53.664167 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:53.664258 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:53.666929 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:53.666934 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:53.666934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:53.666946 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:53.669135 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:53.669927 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:53.669929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:53.669930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:53.670013 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:53.670383 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:53.670384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:53.670385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:53.670522 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:53.670664 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:53.671032 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 16:29:53.671123 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 16:29:53.671440 7825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:53.658585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:36621 (local address 127.30.23.129:37085)
0809 16:29:53.658995 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:53.659037 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:53.660623 (+ 1586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:53.660930 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:53.660934 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:53.661389 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:53.661628 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:53.661633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:53.662860 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:53.662861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:53.663709 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:53.663711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:53.664278 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:53.664279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:53.664408 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:53.665126 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:53.665135 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:53.666761 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:53.669257 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:53.669266 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:53.669268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:53.669784 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:53.670127 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:53.670128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:53.670128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:53.670260 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:53.670651 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:53.670658 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:53.670825 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 16:29:53.670948 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 16:29:53.671164 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":99,"threads_started":1}
I20260809 16:29:53.672271 7821 heartbeater.cc:347] Connected to a master server at 127.30.23.190:36621
I20260809 16:29:53.672375 7821 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:53.672573 7821 heartbeater.cc:511] Master 127.30.23.190:36621 requested a full tablet report, sending...
I20260809 16:29:53.673190 7490 ts_manager.cc:194] Registered new tserver with Master: fbcfd81eb1404cc89610bc0c64b3d52e (127.30.23.129:39503)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
I20260809 16:29:53.674117 7490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:37085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:29:53.695050 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:36621
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:53.787439 7828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:53.787711 7828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:53.787799 7828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:53.789076 7828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:53.789149 7828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:53.789196 7828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:29:53.790869 7828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:36621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:53.791172 7828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:53.791424 7828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:53.797516 7836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:53.797531 7828 server_base.cc:1034] running on GCE node
W20260809 16:29:53.797497 7840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:53.797497 7837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:53.797983 7828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:53.798290 7828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:53.799484 7828 hybrid_clock.cc:648] HybridClock initialized: now 1786292993799432 us; error 78 us; skew 500 ppm
Aug 09 16:29:53 dist-test-slave-j2vl krb5kdc[7409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292993, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:53.803037 7828 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:29:53.803630 7828 webserver.cc:460] Webserver started at http://127.30.23.130:37703/ using document root <none> and password file <none>
I20260809 16:29:53.803876 7828 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:53.803944 7828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:53.804102 7828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:53.805469 7828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "44f84159715f49028910697a37c1fb6f"
format_stamp: "Formatted at 2026-08-09 16:29:53 on dist-test-slave-j2vl"
server_key: "46f4963839a0d3a3265620bdec9c0b50"
server_key_iv: "a7f9c75f003050838d6b8bc78a4617e6"
server_key_version: "encryptionkey@0"
I20260809 16:29:53.805933 7828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "44f84159715f49028910697a37c1fb6f"
format_stamp: "Formatted at 2026-08-09 16:29:53 on dist-test-slave-j2vl"
server_key: "46f4963839a0d3a3265620bdec9c0b50"
server_key_iv: "a7f9c75f003050838d6b8bc78a4617e6"
server_key_version: "encryptionkey@0"
I20260809 16:29:53.807868 7828 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 16:29:53.809036 7848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:53.809430 7828 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:53.809562 7828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "44f84159715f49028910697a37c1fb6f"
format_stamp: "Formatted at 2026-08-09 16:29:53 on dist-test-slave-j2vl"
server_key: "46f4963839a0d3a3265620bdec9c0b50"
server_key_iv: "a7f9c75f003050838d6b8bc78a4617e6"
server_key_version: "encryptionkey@0"
I20260809 16:29:53.809659 7828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:53.839269 7828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:53.841903 7828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:53.842089 7828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:53.842365 7828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:53.842712 7828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:53.842767 7828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:53.842810 7828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:53.842844 7828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:53.849395 7828 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:39479
I20260809 16:29:53.849423 8032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:39479 every 8 connection(s)
I20260809 16:29:53.849794 7828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:29:53.852422 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 7828
I20260809 16:29:53.852537 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:29:53.852741 30814 external_mini_cluster.cc:1428] Setting key 6cdebc12138af9890c7c0a97c6b6217a
Aug 09 16:29:53 dist-test-slave-j2vl krb5kdc[7409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292993, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:53.862191 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:53.850619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:60837 (local address 127.30.23.190:36621)
0809 16:29:53.850672 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:53.850674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:53.850887 (+ 213us) server_negotiation.cc:407] Connection header received
0809 16:29:53.851875 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:53.851876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:53.851888 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:53.851977 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:53.852980 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:53.853689 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:53.855274 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:53.855527 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:53.855605 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:53.858564 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:53.858565 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:53.858565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:53.858573 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:53.860435 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:53.861018 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:53.861019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:53.861019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:53.861078 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:53.861473 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:53.861474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:53.861474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:53.861596 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:53.861655 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:53.862046 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 16:29:53.862121 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:29:53.862515 8041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:53.850568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:36621 (local address 127.30.23.130:60837)
0809 16:29:53.850741 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:53.850786 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:53.851747 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:53.852047 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:53.852050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:53.852430 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:53.852740 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:53.852746 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:53.853782 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:53.853783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:53.855179 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:53.855183 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:53.855616 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:53.855618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:53.855778 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:53.856485 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:53.856494 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:53.858441 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:53.860530 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:53.860540 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:53.860543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:53.860920 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:53.861148 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:53.861149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:53.861150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:53.861318 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:53.861669 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:53.861677 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:53.861874 (+ 197us) client_negotiation.cc:769] Sending connection context
0809 16:29:53.862016 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 16:29:53.862216 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
I20260809 16:29:53.863056 8034 heartbeater.cc:347] Connected to a master server at 127.30.23.190:36621
I20260809 16:29:53.863180 8034 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:53.863373 8034 heartbeater.cc:511] Master 127.30.23.190:36621 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
I20260809 16:29:53.863886 7490 ts_manager.cc:194] Registered new tserver with Master: 44f84159715f49028910697a37c1fb6f (127.30.23.130:39479)
I20260809 16:29:53.864264 7490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:60837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:29:53.885713 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:36621
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:53.970494 8044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:53.970753 8044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:53.970795 8044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:53.972055 8044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:53.972128 8044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:53.972177 8044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:29:53.973836 8044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:36621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:53.974124 8044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:53.974365 8044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:53.979774 8055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:53.979790 8056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:53.979790 8058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:53.980135 8044 server_base.cc:1034] running on GCE node
I20260809 16:29:53.980363 8044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:53.980623 8044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:53.981776 8044 hybrid_clock.cc:648] HybridClock initialized: now 1786292993981764 us; error 35 us; skew 500 ppm
Aug 09 16:29:53 dist-test-slave-j2vl krb5kdc[7409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292993, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:53.984490 8044 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:29:53.984936 8044 webserver.cc:460] Webserver started at http://127.30.23.131:39075/ using document root <none> and password file <none>
I20260809 16:29:53.985179 8044 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:53.985247 8044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:53.985427 8044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:53.986526 8044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "368f7ca0969d4c5badbe765d656dad87"
format_stamp: "Formatted at 2026-08-09 16:29:53 on dist-test-slave-j2vl"
server_key: "08db11511900c0c8e0382f99c80f5139"
server_key_iv: "6f2a6cc5846fe618863cbe1ab3c16244"
server_key_version: "encryptionkey@0"
I20260809 16:29:53.986907 8044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "368f7ca0969d4c5badbe765d656dad87"
format_stamp: "Formatted at 2026-08-09 16:29:53 on dist-test-slave-j2vl"
server_key: "08db11511900c0c8e0382f99c80f5139"
server_key_iv: "6f2a6cc5846fe618863cbe1ab3c16244"
server_key_version: "encryptionkey@0"
I20260809 16:29:53.988348 8044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:29:53.989142 8069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:53.989375 8044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:53.989509 8044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "368f7ca0969d4c5badbe765d656dad87"
format_stamp: "Formatted at 2026-08-09 16:29:53 on dist-test-slave-j2vl"
server_key: "08db11511900c0c8e0382f99c80f5139"
server_key_iv: "6f2a6cc5846fe618863cbe1ab3c16244"
server_key_version: "encryptionkey@0"
I20260809 16:29:53.989591 8044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:54.018926 8044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:54.021421 8044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:54.021605 8044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:54.021867 8044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:54.022202 8044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:54.022255 8044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:54.022307 8044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:54.022337 8044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:54.029150 8044 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:33787
I20260809 16:29:54.029227 8253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:33787 every 8 connection(s)
I20260809 16:29:54.029646 8044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:29:54.032140 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 8044
I20260809 16:29:54.032225 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:29:54.032421 30814 external_mini_cluster.cc:1428] Setting key 22f13b7b332aeae2ca1205b3e2257b13
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[7409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292993, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:29:54.040584 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:54.030539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:45709 (local address 127.30.23.190:36621)
0809 16:29:54.030591 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:54.030594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:54.030772 (+ 178us) server_negotiation.cc:407] Connection header received
0809 16:29:54.031772 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:54.031774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:54.031787 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:54.031842 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:54.032566 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:54.033082 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:54.034165 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:54.034386 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:54.034471 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:54.036935 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:54.036937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:54.036938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:54.036948 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:54.038854 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:54.039384 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:54.039385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:54.039386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:54.039465 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:54.039781 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:54.039782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:54.039783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:54.039929 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:54.040018 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:54.040356 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 16:29:54.040478 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:29:54.040832 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:54.030483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:36621 (local address 127.30.23.131:45709)
0809 16:29:54.030639 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:54.030686 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:54.031659 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:54.031935 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:54.031939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:54.032243 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:54.032485 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:54.032491 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:54.033180 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:54.033181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:54.034071 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:54.034073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:54.034478 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:54.034479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:54.034615 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:54.035154 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:54.035163 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:54.036798 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:54.038971 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:54.038983 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:54.038986 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:54.039285 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:54.039545 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:54.039546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:54.039546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:54.039679 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:54.040005 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:54.040014 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:54.040177 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 16:29:54.040330 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 16:29:54.040530 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260809 16:29:54.041736 8254 heartbeater.cc:347] Connected to a master server at 127.30.23.190:36621
I20260809 16:29:54.041882 8254 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:54.042061 8254 heartbeater.cc:511] Master 127.30.23.190:36621 requested a full tablet report, sending...
I20260809 16:29:54.042544 7490 ts_manager.cc:194] Registered new tserver with Master: 368f7ca0969d4c5badbe765d656dad87 (127.30.23.131:33787)
I20260809 16:29:54.042972 7490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:45709
I20260809 16:29:54.045305 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/test-admin.keytab.
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[7409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:54.067637 30814 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 16:29:54.069685 8264 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:29:54.068628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.30.23.129:39503)
0809 16:29:54.068879 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:54.068884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:54.068895 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:29:54.068923 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:54.068925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:54.068944 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:54.069066 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:54.069576 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36978: BlockingRecv error: recv got EOF from 127.0.0.1:36978 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":141,"threads_started":1}
W20260809 16:29:54.069777 30814 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/krb5cc' of type FILE
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[7409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:54.070932 30814 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[7409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:54.072095 30814 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[7409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.129@KRBTEST.COM
I20260809 16:29:54.080840 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:54.072709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36992 (local address 127.30.23.129:39503)
0809 16:29:54.072767 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:54.072770 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:54.072831 (+ 61us) server_negotiation.cc:407] Connection header received
0809 16:29:54.072927 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:54.072928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:54.072945 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:54.073007 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:54.073484 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:54.074229 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:54.074970 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:54.075317 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:54.075397 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:54.077733 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:54.077744 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:54.077744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:54.077753 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:54.079424 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:54.079783 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:54.079792 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:54.079792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:54.079846 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:54.080110 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:54.080111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:54.080111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:54.080352 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:54.080538 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:54.080630 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 16:29:54.080722 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260809 16:29:54.082890 8264 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:29:54.082200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36998 (local address 127.30.23.129:39503)
0809 16:29:54.082236 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:54.082238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:54.082309 (+ 71us) server_negotiation.cc:407] Connection header received
0809 16:29:54.082365 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:54.082366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:54.082374 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:54.082427 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:54.082825 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36998: BlockingRecv error: recv got EOF from 127.0.0.1:36998 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:29:54.083015 30814 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/krb5cc' of type FILE
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[7409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:54.084067 30814 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[7409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:54.085096 30814 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[7409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.129@KRBTEST.COM
I20260809 16:29:54.092978 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:54.085844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37012 (local address 127.30.23.129:39503)
0809 16:29:54.085899 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:54.085901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:54.085911 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:29:54.085937 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:54.085938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:54.085945 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:54.086000 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:54.086454 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:54.086988 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:54.087707 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:54.087926 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:54.087981 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:54.090314 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:54.090320 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:54.090321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:54.090328 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:54.091758 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:54.092181 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:54.092182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:54.092183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:54.092241 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:54.092512 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:54.092513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:54.092513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:54.092624 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:54.092714 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:54.092798 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 16:29:54.092870 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 16:29:54.094583 8264 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:29:54.094023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37016 (local address 127.30.23.129:39503)
0809 16:29:54.094080 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:54.094082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:54.094114 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:29:54.094185 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:54.094186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:54.094195 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:54.094262 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:54.094532 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37016: BlockingRecv error: recv got EOF from 127.0.0.1:37016 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 16:29:54.094666 30814 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786292963539133-30814-0/krb5kdc/krb5cc' of type FILE
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[7409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:54.095742 30814 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[7409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:54.096765 30814 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[7409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.129@KRBTEST.COM
I20260809 16:29:54.106045 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:54.097326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37030 (local address 127.30.23.129:39503)
0809 16:29:54.097452 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:54.097454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:54.097466 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:29:54.097566 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:54.097567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:54.097578 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:54.097639 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:54.098140 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:54.098920 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:54.100080 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:54.100399 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:54.100477 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:54.103115 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:54.103116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:54.103117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:54.103143 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:54.104745 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:54.105154 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:54.105155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:54.105155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:54.105228 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:54.105545 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:54.105546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:54.105546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:54.105669 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:54.105784 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:54.105888 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 16:29:54.105970 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 16:29:54.107223 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 7587
I20260809 16:29:54.119767 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 7828
I20260809 16:29:54.134459 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 8044
I20260809 16:29:54.181048 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 7437
2026-08-09T16:29:54Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2316 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[8281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[8281](info): set up 2 sockets
Aug 09 16:29:54 dist-test-slave-j2vl krb5kdc[8281](info): commencing operation
krb5kdc: starting...
W20260809 16:29:55.512492 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:29:55 dist-test-slave-j2vl krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:29:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:29:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.190.keytab.
Entry for principal oryx/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.190.keytab.
I20260809 16:29:55.649860 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:35975
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.190.keytab
--principal=oryx/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:35975
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:55.736912 8302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:55.737212 8302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:55.737253 8302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:55.738564 8302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:55.738644 8302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:55.738670 8302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:55.738692 8302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:55.738722 8302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:55.740324 8302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:35975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.190.keytab
--principal=oryx/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:35975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:55.740639 8302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:55.740883 8302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:55.746256 8311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:55.746320 8317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:55.746271 8312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:55.746577 8302 server_base.cc:1034] running on GCE node
I20260809 16:29:55.746874 8302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:55.747200 8302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:55.748401 8302 hybrid_clock.cc:648] HybridClock initialized: now 1786292995748388 us; error 31 us; skew 500 ppm
Aug 09 16:29:55 dist-test-slave-j2vl krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292995, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:55.755596 8302 init.cc:377] Logged in from keytab as oryx/127.30.23.190@KRBTEST.COM (short username oryx)
I20260809 16:29:55.756117 8302 webserver.cc:460] Webserver started at http://127.30.23.190:41165/ using document root <none> and password file <none>
I20260809 16:29:55.756358 8302 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:55.756428 8302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:55.756588 8302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:55.757827 8302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3"
format_stamp: "Formatted at 2026-08-09 16:29:55 on dist-test-slave-j2vl"
server_key: "9ef191d790716725e6763d9c7036ff15"
server_key_iv: "8f2487a2bdde130178bfc42c33c33e67"
server_key_version: "encryptionkey@0"
I20260809 16:29:55.758253 8302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3"
format_stamp: "Formatted at 2026-08-09 16:29:55 on dist-test-slave-j2vl"
server_key: "9ef191d790716725e6763d9c7036ff15"
server_key_iv: "8f2487a2bdde130178bfc42c33c33e67"
server_key_version: "encryptionkey@0"
I20260809 16:29:55.759986 8302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:29:55.760946 8328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:55.761175 8302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:55.761304 8302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3"
format_stamp: "Formatted at 2026-08-09 16:29:55 on dist-test-slave-j2vl"
server_key: "9ef191d790716725e6763d9c7036ff15"
server_key_iv: "8f2487a2bdde130178bfc42c33c33e67"
server_key_version: "encryptionkey@0"
I20260809 16:29:55.761451 8302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:55.783113 8302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:55.785233 8302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:55.785446 8302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:55.789852 8302 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:35975
I20260809 16:29:55.789927 8415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:35975 every 8 connection(s)
I20260809 16:29:55.790299 8302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:29:55.791369 8416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:55.793978 8416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3: Bootstrap starting.
I20260809 16:29:55.794595 8416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:55.794902 8416 log.cc:824] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:55.795643 8416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3: No bootstrap required, opened a new log
I20260809 16:29:55.796164 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 8302
I20260809 16:29:55.796288 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:29:55.796514 30814 external_mini_cluster.cc:1428] Setting key b4dbbbfdba5b4d0fcc5c17b65a1cd53f
I20260809 16:29:55.797834 8416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } }
I20260809 16:29:55.797997 8416 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:55.798049 8416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1d69d1116e40e29f5dfb7ec05c37b3, State: Initialized, Role: FOLLOWER
I20260809 16:29:55.798179 8416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } }
I20260809 16:29:55.798264 8416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:29:55.798310 8416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:29:55.798377 8416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:55.799536 8416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } }
I20260809 16:29:55.799687 8416 leader_election.cc:302] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d1d69d1116e40e29f5dfb7ec05c37b3; no voters:
I20260809 16:29:55.799921 8416 leader_election.cc:288] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:29:55.799945 8424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:55.800088 8424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [term 1 LEADER]: Becoming Leader. State: Replica: 1d1d69d1116e40e29f5dfb7ec05c37b3, State: Running, Role: LEADER
I20260809 16:29:55.800185 8424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } }
I20260809 16:29:55.800225 8416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:55.800735 8426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d1d69d1116e40e29f5dfb7ec05c37b3. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } }
I20260809 16:29:55.800835 8426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:55.801081 8433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:29:55.801980 8433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:29:55.802477 8427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } }
I20260809 16:29:55.802652 8427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:55.803819 8433 catalog_manager.cc:1347] Generated new cluster ID: b0b8579f5eec4a569d041b1164b4e8b0
I20260809 16:29:55.803884 8433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:29:55 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:29:55.809559 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:55.797082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44490 (local address 127.30.23.190:35975)
0809 16:29:55.797299 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:55.797307 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:55.797333 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:29:55.797808 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:55.797815 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:55.797890 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:55.798024 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:55.798936 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:55.800042 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:55.801553 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:55.802443 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:55.802563 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:55.805663 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:55.805668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:55.805668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:55.805685 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:55.807449 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:55.807841 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:55.807852 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:55.807852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:55.807964 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:55.808180 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:55.808181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:55.808182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:55.808441 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:55.809160 (+ 719us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:55.809222 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 16:29:55.809337 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":112,"threads_started":1}
I20260809 16:29:55.842144 8433 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:29:55.842756 8433 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:29:55.851444 8433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3: Generated new TSK 0
I20260809 16:29:55.851670 8433 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab.
Entry for principal oryx/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab.
I20260809 16:29:55.893054 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:35975
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab
--principal=oryx/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:56.009217 8456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:56.009487 8456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:56.009516 8456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:56.010969 8456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:56.011031 8456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:56.011137 8456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:29:56.012856 8456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab
--principal=oryx/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:35975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:56.013157 8456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:56.013583 8456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:56.019049 8469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:56.019030 8466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:56.019030 8467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:56.019618 8456 server_base.cc:1034] running on GCE node
I20260809 16:29:56.019837 8456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:56.020115 8456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:56.021425 8456 hybrid_clock.cc:648] HybridClock initialized: now 1786292996021416 us; error 45 us; skew 500 ppm
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:56.024399 8456 init.cc:377] Logged in from keytab as oryx/127.30.23.129@KRBTEST.COM (short username oryx)
I20260809 16:29:56.024927 8456 webserver.cc:460] Webserver started at http://127.30.23.129:38995/ using document root <none> and password file <none>
I20260809 16:29:56.025177 8456 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:56.025246 8456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:56.025518 8456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:56.026618 8456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "3e682250219344f09ef1b9bd92b8a2f2"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "c43af381a2a0c806b281cae9d15ca261"
server_key_iv: "4b7bd7dd33702d741c7fd1e1f086df0c"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.027019 8456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "3e682250219344f09ef1b9bd92b8a2f2"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "c43af381a2a0c806b281cae9d15ca261"
server_key_iv: "4b7bd7dd33702d741c7fd1e1f086df0c"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.028482 8456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:29:56.029457 8478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.029680 8456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:56.029839 8456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "3e682250219344f09ef1b9bd92b8a2f2"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "c43af381a2a0c806b281cae9d15ca261"
server_key_iv: "4b7bd7dd33702d741c7fd1e1f086df0c"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.029928 8456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:56.042263 8456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:56.045190 8456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:56.045517 8456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:56.045920 8456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:56.046305 8456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:56.046455 8456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.046546 8456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:56.046584 8456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.054579 8456 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:33891
I20260809 16:29:56.054674 8669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:33891 every 8 connection(s)
I20260809 16:29:56.055171 8456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:29:56.060729 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 8456
I20260809 16:29:56.060858 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:29:56.061080 30814 external_mini_cluster.cc:1428] Setting key ee10d9ab888ae22c98abe0c3fb76884b
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:29:56.068549 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.056028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.129:60031)
0809 16:29:56.056225 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.056268 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.056794 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.057118 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.057122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.057791 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.057998 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.058003 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.058860 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.058862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.059966 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.059970 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.060528 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.060529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.060713 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.061448 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.061458 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.063847 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:56.066704 (+ 2857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.066714 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.066716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.067005 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.067261 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.067262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.067262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.067480 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.067942 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:56.067955 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:56.068101 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 16:29:56.068198 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 16:29:56.068360 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":101,"threads_started":1}
I20260809 16:29:56.068609 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.056084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:60031 (local address 127.30.23.190:35975)
0809 16:29:56.056159 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.056162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.056355 (+ 193us) server_negotiation.cc:407] Connection header received
0809 16:29:56.056914 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.056916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.056933 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.057002 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.058114 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.058727 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.060085 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.060308 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.060383 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.064072 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.064074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.064075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.064086 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.066583 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.067091 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.067092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.067093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.067164 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.067577 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.067579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.067579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.067764 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.067923 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.068297 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.068439 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:29:56.069187 8670 heartbeater.cc:347] Connected to a master server at 127.30.23.190:35975
I20260809 16:29:56.069332 8670 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:56.069631 8670 heartbeater.cc:511] Master 127.30.23.190:35975 requested a full tablet report, sending...
I20260809 16:29:56.070446 8353 ts_manager.cc:194] Registered new tserver with Master: 3e682250219344f09ef1b9bd92b8a2f2 (127.30.23.129:33891)
I20260809 16:29:56.071086 8353 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:60031
WARNING: no policy specified for oryx/127.30.23.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab.
Entry for principal oryx/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab.
I20260809 16:29:56.098417 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:35975
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab
--principal=oryx/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:56.187856 8679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:56.188074 8679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:56.188098 8679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:56.189314 8679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:56.189405 8679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:56.189452 8679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:29:56.190901 8679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab
--principal=oryx/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:35975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:56.191092 8679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:56.191278 8679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:56.196643 8690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:56.196663 8685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:56.196668 8686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:56.196707 8679 server_base.cc:1034] running on GCE node
I20260809 16:29:56.197016 8679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:56.197263 8679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:56.198454 8679 hybrid_clock.cc:648] HybridClock initialized: now 1786292996198438 us; error 29 us; skew 500 ppm
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:56.201465 8679 init.cc:377] Logged in from keytab as oryx/127.30.23.130@KRBTEST.COM (short username oryx)
I20260809 16:29:56.201941 8679 webserver.cc:460] Webserver started at http://127.30.23.130:34871/ using document root <none> and password file <none>
I20260809 16:29:56.202174 8679 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:56.202242 8679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:56.202379 8679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:56.203469 8679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "110100e55f1e404bba5a298ef2699738"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "296b28c6d9d24665e800017243f06a72"
server_key_iv: "59b17622dfa46932f385ac2ea50ff7ce"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.203840 8679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "110100e55f1e404bba5a298ef2699738"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "296b28c6d9d24665e800017243f06a72"
server_key_iv: "59b17622dfa46932f385ac2ea50ff7ce"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.205384 8679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:29:56.206195 8701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.206394 8679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:56.206497 8679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "110100e55f1e404bba5a298ef2699738"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "296b28c6d9d24665e800017243f06a72"
server_key_iv: "59b17622dfa46932f385ac2ea50ff7ce"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.206578 8679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:56.224720 8679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:56.226974 8679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:56.227150 8679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:56.227406 8679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:56.227730 8679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:56.227783 8679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.227826 8679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:56.227871 8679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.233927 8679 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:45517
I20260809 16:29:56.234027 8899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:45517 every 8 connection(s)
I20260809 16:29:56.234342 8679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:29:56.235308 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 8679
I20260809 16:29:56.235455 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:29:56.235683 30814 external_mini_cluster.cc:1428] Setting key 034102ecf3f86c4fc22a2b5869da4058
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:29:56.245535 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.235118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:41453 (local address 127.30.23.190:35975)
0809 16:29:56.235191 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.235194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.235386 (+ 192us) server_negotiation.cc:407] Connection header received
0809 16:29:56.235995 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.235997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.236014 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.236087 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.237447 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.238008 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.239195 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.239578 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.239666 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.242391 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.242392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.242393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.242400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.243826 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.244389 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.244389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.244390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.244454 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.244761 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.244761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.244761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.244866 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.244917 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.245300 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.245467 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:29:56.245821 8903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.235084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.130:41453)
0809 16:29:56.235250 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.235294 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.235888 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.236161 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.236166 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.237027 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.237321 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.237327 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.238110 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.238111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.239104 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.239107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.239670 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.239671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.239771 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.240342 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.240351 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.242286 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:56.243918 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.243927 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.243930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.244307 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.244526 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.244527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.244527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.244689 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.244935 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:56.244943 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:56.245138 (+ 195us) client_negotiation.cc:769] Sending connection context
0809 16:29:56.245294 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 16:29:56.245544 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for oryx/127.30.23.131@KRBTEST.COM; defaulting to no policy
I20260809 16:29:56.246414 8900 heartbeater.cc:347] Connected to a master server at 127.30.23.190:35975
I20260809 16:29:56.246513 8900 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:56.246668 8900 heartbeater.cc:511] Master 127.30.23.190:35975 requested a full tablet report, sending...
I20260809 16:29:56.247177 8353 ts_manager.cc:194] Registered new tserver with Master: 110100e55f1e404bba5a298ef2699738 (127.30.23.130:45517)
I20260809 16:29:56.247565 8353 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:41453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab.
Entry for principal oryx/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab.
I20260809 16:29:56.266840 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:35975
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab
--principal=oryx/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:56.355341 8910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:56.355556 8910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:56.355580 8910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:56.356686 8910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:56.356724 8910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:56.356755 8910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:29:56.358253 8910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab
--principal=oryx/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:35975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:56.358449 8910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:56.358634 8910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:56.363866 8919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:56.363907 8910 server_base.cc:1034] running on GCE node
W20260809 16:29:56.363875 8922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:56.363889 8916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:56.364297 8910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:56.364591 8910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:56.365787 8910 hybrid_clock.cc:648] HybridClock initialized: now 1786292996365775 us; error 38 us; skew 500 ppm
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:56.368924 8910 init.cc:377] Logged in from keytab as oryx/127.30.23.131@KRBTEST.COM (short username oryx)
I20260809 16:29:56.369590 8910 webserver.cc:460] Webserver started at http://127.30.23.131:43795/ using document root <none> and password file <none>
I20260809 16:29:56.370241 8910 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:56.370602 8910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:56.370769 8910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:56.371969 8910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "0a81ae62aedb4143a7ecd70c9e9742fb"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "a0fad6d05c29f3893fcf219b8102e46e"
server_key_iv: "7c11882dacc8daff66c376e1befefcfc"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.372361 8910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "0a81ae62aedb4143a7ecd70c9e9742fb"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "a0fad6d05c29f3893fcf219b8102e46e"
server_key_iv: "7c11882dacc8daff66c376e1befefcfc"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.374017 8910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260809 16:29:56.374890 8939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.375157 8910 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:56.375290 8910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "0a81ae62aedb4143a7ecd70c9e9742fb"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "a0fad6d05c29f3893fcf219b8102e46e"
server_key_iv: "7c11882dacc8daff66c376e1befefcfc"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.375371 8910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:56.392746 8910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:56.395478 8910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:56.395639 8910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:56.395910 8910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:56.396255 8910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:56.396291 8910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.396332 8910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:56.396359 8910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.402829 8910 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:35311
I20260809 16:29:56.402874 9150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:35311 every 8 connection(s)
I20260809 16:29:56.403203 8910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:29:56.413390 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 8910
I20260809 16:29:56.413514 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:29:56.413542 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.404019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:54807 (local address 127.30.23.190:35975)
0809 16:29:56.404109 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.404111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.404312 (+ 201us) server_negotiation.cc:407] Connection header received
0809 16:29:56.404850 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.404851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.404860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.404925 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.406038 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.406556 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.407595 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.407805 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.407873 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.410219 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.410221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.410221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.410229 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.411824 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.412280 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.412281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.412281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.412350 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.412675 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.412677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.412677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.412849 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.412960 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.413313 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.413448 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 16:29:56.413743 30814 external_mini_cluster.cc:1428] Setting key 8ad0fcfa7603d9a315e50bb1ab28ce44
I20260809 16:29:56.413738 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.403975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.131:54807)
0809 16:29:56.404175 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.404221 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.404761 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.405060 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.405064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.405743 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.405944 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.405949 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.406642 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.406643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.407479 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.407481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.407902 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.407903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.408042 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.408547 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.408557 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.410123 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:56.411936 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.411944 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.411946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.412190 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.412441 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.412442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.412443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.412569 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.412950 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:56.412956 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:56.413103 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 16:29:56.413261 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 16:29:56.413472 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":112,"threads_started":1}
I20260809 16:29:56.414376 9151 heartbeater.cc:347] Connected to a master server at 127.30.23.190:35975
I20260809 16:29:56.414502 9151 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:56.414738 9151 heartbeater.cc:511] Master 127.30.23.190:35975 requested a full tablet report, sending...
I20260809 16:29:56.415153 8353 ts_manager.cc:194] Registered new tserver with Master: 0a81ae62aedb4143a7ecd70c9e9742fb (127.30.23.131:35311)
I20260809 16:29:56.415814 8353 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:54807
I20260809 16:29:56.426162 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.23.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.23.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.23.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.189.keytab.
Entry for principal oryx/127.30.23.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.189.keytab.
I20260809 16:29:56.457237 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.189:35459
--webserver_interface=127.30.23.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.189.keytab
--principal=oryx/127.30.23.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:35975,127.30.23.189:35459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:56.547322 9163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:56.547555 9163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:56.547580 9163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:56.548846 9163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:56.548895 9163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:56.548911 9163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:56.548924 9163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:56.548938 9163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:56.550558 9163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:35975,127.30.23.189:35459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.189.keytab
--principal=oryx/127.30.23.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.189:35459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.23.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
W20260809 16:29:56.550675 9163 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.23.190:35975,127.30.23.189:35459'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 16:29:56.550861 9163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:56.551110 9163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:56.556416 9177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:56.556408 9174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:56.556602 9175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:56.556746 9163 server_base.cc:1034] running on GCE node
I20260809 16:29:56.557003 9163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:56.557322 9163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:56.558530 9163 hybrid_clock.cc:648] HybridClock initialized: now 1786292996558518 us; error 34 us; skew 500 ppm
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:56.561236 9163 init.cc:377] Logged in from keytab as oryx/127.30.23.189@KRBTEST.COM (short username oryx)
I20260809 16:29:56.561884 9163 webserver.cc:460] Webserver started at http://127.30.23.189:40677/ using document root <none> and password file <none>
I20260809 16:29:56.562266 9163 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:56.562335 9163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:56.562565 9163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:56.564018 9163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/data/instance:
uuid: "8d9c32a955cf44c091f52611fe665054"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "b02e19ea098e292c4622c84f6f58c1bd"
server_key_iv: "8a5cdccd7f7363487d950b30fff92115"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.564554 9163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/wal/instance:
uuid: "8d9c32a955cf44c091f52611fe665054"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "b02e19ea098e292c4622c84f6f58c1bd"
server_key_iv: "8a5cdccd7f7363487d950b30fff92115"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.566169 9163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:56.566947 9190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.567123 9163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:29:56.567214 9163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/wal
uuid: "8d9c32a955cf44c091f52611fe665054"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "b02e19ea098e292c4622c84f6f58c1bd"
server_key_iv: "8a5cdccd7f7363487d950b30fff92115"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.567269 9163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:56.578866 9163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:56.581099 9163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:56.581224 9163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:56.585593 9163 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.189:35459
I20260809 16:29:56.585738 9287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.189:35459 every 8 connection(s)
I20260809 16:29:56.586061 9163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/data/info.pb
I20260809 16:29:56.587262 9290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:56.588861 9290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:29:56.593523 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 9163
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.189@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:29:56.598872 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.589711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44494 (local address 127.30.23.190:35975)
0809 16:29:56.589811 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.589813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.590038 (+ 225us) server_negotiation.cc:407] Connection header received
0809 16:29:56.590552 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.590553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.590567 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.590633 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.591314 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.591821 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.592831 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.593033 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.593118 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.595558 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.595560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.595560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.595567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.597085 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.597664 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.597666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.597666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.597734 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.598102 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.598104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.598104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.598280 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.598398 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.598728 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.598805 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 16:29:56.599123 9294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.589684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:44494)
0809 16:29:56.589911 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.589949 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.590447 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.590705 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.590708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.590984 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.591200 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.591204 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.591915 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.591916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.592739 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.592741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.593126 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.593128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.593249 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.593852 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.593867 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.595455 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:56.597194 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.597201 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.597204 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.597580 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.597803 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.597804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.597805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.598006 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.598386 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:56.598393 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:56.598569 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 16:29:56.598697 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 16:29:56.598882 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":135,"threads_started":1}
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.23.189@KRBTEST.COM
I20260809 16:29:56.599857 9290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:29:56.603039 9296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.594127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59316 (local address 127.30.23.189:35459)
0809 16:29:56.594300 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.594306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.594330 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:29:56.594388 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.594391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.594414 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.594497 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.595130 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.595947 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.596699 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.597009 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.597089 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.599696 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.599702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.599702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.599711 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.601806 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.602245 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.602248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.602249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.602319 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.602518 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.602518 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.602518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.602640 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.602812 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.602857 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.602935 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.189@KRBTEST.COM for oryx/127.30.23.189@KRBTEST.COM
I20260809 16:29:56.608050 9294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.600172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:35459 (local address 127.0.0.1:59330)
0809 16:29:56.600206 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.600217 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.600256 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.600540 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.600541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.600707 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.600776 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.600778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.601865 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.601867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.602636 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.602639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.603118 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.603119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.603184 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.603510 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.603515 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.604724 (+ 1209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:56.606976 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.606976 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.606977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.607148 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.607439 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.607440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.607441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.607509 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.607760 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:56.607761 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:56.607823 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 16:29:56.607884 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 16:29:56.607977 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 16:29:56.608201 9297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.600214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.30.23.189:35459)
0809 16:29:56.600426 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.600429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.600436 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:29:56.600454 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.600455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.600464 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.600536 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.600867 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.601740 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.602720 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.603029 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.603327 (+ 298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.604813 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.604814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.604815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.604821 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.606869 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.607245 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.607246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.607246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.607345 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.607578 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.607579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.607579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.607686 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.607764 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.607903 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.608025 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":191,"thread_start_us":117,"threads_started":1}
I20260809 16:29:56.609747 9290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: Bootstrap starting.
I20260809 16:29:56.610525 9290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:56.610785 9290 log.cc:824] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:56.611572 9290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: No bootstrap required, opened a new log
I20260809 16:29:56.613036 9290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } }
I20260809 16:29:56.613164 9290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:56.613188 9290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d9c32a955cf44c091f52611fe665054, State: Initialized, Role: FOLLOWER
I20260809 16:29:56.613312 9290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } }
I20260809 16:29:56.613664 9299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } } }
I20260809 16:29:56.613740 9290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:56.613795 9299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:29:56.615485 9315 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:29:56.615550 9315 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:29:56.617308 9163 master_runner.cc:428] Detected that this master 8d9c32a955cf44c091f52611fe665054 is joining an existing cluster
I20260809 16:29:56.617444 9163 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.23.189:35459
I20260809 16:29:56.625969 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.618492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:35459 (local address 127.0.0.1:59342)
0809 16:29:56.618689 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.618699 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.618757 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.618932 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.618933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.619128 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.619191 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.619193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.620108 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.620109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.621038 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.621040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.621468 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.621469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.621540 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.622367 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.622372 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.622811 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:56.624675 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.624676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.624677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.625033 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.625220 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.625221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.625221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.625293 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.625605 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:56.625606 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:56.625660 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 16:29:56.625715 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:29:56.625833 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":27,"thread_start_us":91,"threads_started":1}
I20260809 16:29:56.626026 9297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.618681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59342 (local address 127.30.23.189:35459)
0809 16:29:56.618716 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.618718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.618742 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:29:56.618834 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.618834 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.618843 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.618897 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.619265 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.620021 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.621126 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.621395 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.621463 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.622906 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.622907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.622908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.622913 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.624573 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.625116 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.625117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.625118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.625171 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.625388 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.625388 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.625389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.625540 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.625575 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.625752 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.625904 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":20}
I20260809 16:29:56.626067 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.618506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44510 (local address 127.30.23.190:35975)
0809 16:29:56.618559 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.618561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.618686 (+ 125us) server_negotiation.cc:407] Connection header received
0809 16:29:56.618800 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.618801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.618813 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.618870 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.619315 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.620060 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.620845 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.621054 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.621137 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.622749 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.622751 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.622751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.622759 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.624652 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.625121 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.625122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.625123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.625192 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.625437 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.625438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.625439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.625592 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.625662 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.625879 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.625973 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:29:56.626466 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.618462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:44510)
0809 16:29:56.618620 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.618632 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.618713 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.618875 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.618876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.619131 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.619201 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.619203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.620152 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.620153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.620755 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.620757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.621443 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.621444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.621499 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.622120 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.622126 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.622632 (+ 506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:56.624737 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.624738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.624738 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.625033 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.625266 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.625266 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.625267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.625337 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.625693 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:56.625694 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:56.625757 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 16:29:56.626238 (+ 481us) client_negotiation.cc:240] Negotiation successful
0809 16:29:56.626345 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":39,"thread_start_us":104,"threads_started":1}
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292995, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.190@KRBTEST.COM for oryx/127.30.23.189@KRBTEST.COM
I20260809 16:29:56.637269 9297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.629049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.30.23.189:35459)
0809 16:29:56.629101 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.629103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.629250 (+ 147us) server_negotiation.cc:407] Connection header received
0809 16:29:56.629380 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.629381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.629393 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.629457 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.629957 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.630517 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.631412 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.631601 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.631672 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.634064 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.634066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.634066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.634074 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.635561 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.636200 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.636202 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.636202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.636276 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.636615 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.636617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.636617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.636755 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.636845 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.637075 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.637167 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:29:56.637277 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.628946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:35459 (local address 127.0.0.1:59348)
0809 16:29:56.629120 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.629169 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.629281 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.629466 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.629469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.629796 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.629865 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.629867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.630662 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.630663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.631315 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.631318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.631673 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.631674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.631756 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.632298 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.632302 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.633971 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:56.635670 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.635679 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.635682 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.636080 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.636370 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.636371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.636372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.636522 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.636849 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:56.636858 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:56.636931 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 16:29:56.637044 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 16:29:56.637180 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
I20260809 16:29:56.638064 8353 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.23.189:35459: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8d9c32a955cf44c091f52611fe665054"
member_type: NON_VOTER
last_known_addr {
host: "127.30.23.189"
port: 35459
}
attrs {
promote: true
}
}
dest_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3"
cas_config_opid_index: -1
I20260809 16:29:56.638422 8353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true } }
I20260809 16:29:56.638993 9328 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8d9c32a955cf44c091f52611fe665054 (127.30.23.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true } } }
I20260809 16:29:56.639369 9239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:56.640070 9328 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.23.189:35459
I20260809 16:29:56.640377 9239 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Refusing update from remote peer 1d1d69d1116e40e29f5dfb7ec05c37b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 16:29:56.640411 9327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true } } }
I20260809 16:29:56.640503 9327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:56.640542 9163 master.cc:537] Master@127.30.23.189:35459 shutting down...
I20260809 16:29:56.640547 9299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:29:56.640718 9299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } }
I20260809 16:29:56.640930 9299 leader_election.cc:288] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975)
I20260809 16:29:56.641152 8373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d9c32a955cf44c091f52611fe665054" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" is_pre_election: true
I20260809 16:29:56.641335 9327 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:56.641537 9191 leader_election.cc:302] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 8d9c32a955cf44c091f52611fe665054; no voters: 1d1d69d1116e40e29f5dfb7ec05c37b3
I20260809 16:29:56.641726 9299 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 16:29:56.642165 8330 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 16:29:56.642694 8330 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 -> Peer 8d9c32a955cf44c091f52611fe665054 (127.30.23.189:35459): Couldn't send request to peer 8d9c32a955cf44c091f52611fe665054. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 16:29:56.643685 9163 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:56.643788 9163 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:56.643841 9163 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: stopping tablet replica
I20260809 16:29:56.657330 9163 master.cc:552] Master@127.30.23.189:35459 shutdown complete.
I20260809 16:29:56.657444 9163 master_runner.cc:305] Clearing existing system tablet
I20260809 16:29:56.657593 9163 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:56.657645 9163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:56.658301 9163 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:29:56.658803 9340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.658946 9163 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:56.659003 9163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.659021 9163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/wal
uuid: "8d9c32a955cf44c091f52611fe665054"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "b02e19ea098e292c4622c84f6f58c1bd"
server_key_iv: "8a5cdccd7f7363487d950b30fff92115"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.659711 9163 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:56.661115 9163 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 16:29:56.661180 9163 log.cc:1196] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 16:29:56.661514 9163 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: Deleting consensus metadata
I20260809 16:29:56.661654 9163 master_runner.cc:315] Copying system tablet from 127.30.23.190:35975
I20260809 16:29:56.662303 9163 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: tablet copy: Beginning tablet copy session from remote peer at address 127.30.23.190:35975
I20260809 16:29:56.669549 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.662731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:44520)
0809 16:29:56.662849 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.662862 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.662954 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.663102 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.663102 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.663269 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.663326 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.663328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.664317 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.664318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.664930 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.664932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.665547 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.665548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.665624 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.666200 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.666205 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.666527 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:56.668361 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.668362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.668362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.668562 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.668767 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.668767 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.668767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.668818 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.669126 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:56.669128 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:56.669194 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 16:29:56.669273 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 16:29:56.669461 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":74,"threads_started":1}
I20260809 16:29:56.669579 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.662809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44520 (local address 127.30.23.190:35975)
0809 16:29:56.662867 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.662869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.662930 (+ 61us) server_negotiation.cc:407] Connection header received
0809 16:29:56.663004 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.663005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.663017 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.663099 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.663404 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.664214 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.665009 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.665329 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.665543 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.666641 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.666642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.666642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.666652 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.668272 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.668633 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.668634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.668635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.668700 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.668874 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.668875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.668875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.669015 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.669087 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.669314 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.669483 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:29:56.669945 8390 tablet_copy_service.cc:140] P 1d1d69d1116e40e29f5dfb7ec05c37b3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8d9c32a955cf44c091f52611fe665054 ({username='oryx', principal='oryx/127.30.23.189@KRBTEST.COM'} at 127.0.0.1:44520)
I20260809 16:29:56.670042 8390 tablet_copy_service.cc:161] P 1d1d69d1116e40e29f5dfb7ec05c37b3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8d9c32a955cf44c091f52611fe665054 at {username='oryx', principal='oryx/127.30.23.189@KRBTEST.COM'} at 127.0.0.1:44520: session id = 8d9c32a955cf44c091f52611fe665054-00000000000000000000000000000000
I20260809 16:29:56.670737 8390 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 16:29:56.671437 9163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 16:29:56.672775 9163 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: tablet copy: Starting download of 0 data blocks...
I20260809 16:29:56.672940 9163 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: tablet copy: Starting download of 1 WAL segments...
I20260809 16:29:56.673931 9163 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 16:29:56.674978 8390 tablet_copy_service.cc:342] P 1d1d69d1116e40e29f5dfb7ec05c37b3: Request end of tablet copy session 8d9c32a955cf44c091f52611fe665054-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.23.189@KRBTEST.COM'} at 127.0.0.1:44520
I20260809 16:29:56.675042 8390 tablet_copy_service.cc:433] P 1d1d69d1116e40e29f5dfb7ec05c37b3: ending tablet copy session 8d9c32a955cf44c091f52611fe665054-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8d9c32a955cf44c091f52611fe665054
I20260809 16:29:56.677834 9163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:56.679153 9350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:56.679386 9349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:56.679406 9354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:56.681497 9163 server_base.cc:1034] running on GCE node
I20260809 16:29:56.681602 9163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:56.681770 9163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:56.682901 9163 hybrid_clock.cc:648] HybridClock initialized: now 1786292996682893 us; error 27 us; skew 500 ppm
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:56.684384 9163 init.cc:377] Logged in from keytab as oryx/127.30.23.189@KRBTEST.COM (short username oryx)
I20260809 16:29:56.684700 9163 webserver.cc:460] Webserver started at http://127.30.23.189:37297/ using document root <none> and password file <none>
I20260809 16:29:56.684824 9163 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:56.684875 9163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:56.685534 9163 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:29:56.686057 9362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.686228 9163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:29:56.686288 9163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/wal
uuid: "8d9c32a955cf44c091f52611fe665054"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "b02e19ea098e292c4622c84f6f58c1bd"
server_key_iv: "8a5cdccd7f7363487d950b30fff92115"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.686345 9163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:56.715596 9163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:56.717386 9163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:56.720705 9163 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.189:35459
I20260809 16:29:56.720736 9456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.189:35459 every 8 connection(s)
I20260809 16:29:56.721077 9163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-1/data/info.pb
I20260809 16:29:56.721839 9457 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:29:56.722275 9457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: Bootstrap starting.
I20260809 16:29:56.724659 9457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:29:56.724877 9457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: Bootstrap complete.
I20260809 16:29:56.725076 9457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true } }
I20260809 16:29:56.725164 9457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8d9c32a955cf44c091f52611fe665054, State: Initialized, Role: LEARNER
I20260809 16:29:56.725217 9457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true } }
I20260809 16:29:56.725445 9457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:56.725436 9462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true } } }
I20260809 16:29:56.725543 9462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: This master's current role is: LEARNER
I20260809 16:29:56.726972 9480 catalog_manager.cc:1259] Loaded cluster ID: b0b8579f5eec4a569d041b1164b4e8b0
I20260809 16:29:56.727021 9480 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: loading cluster ID for follower catalog manager: success
I20260809 16:29:56.727972 9480 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: acquiring CA information for follower catalog manager: success
I20260809 16:29:56.728260 9480 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:29:56.761894 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.755322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59372 (local address 127.30.23.189:35459)
0809 16:29:56.755493 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.755495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.755502 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:29:56.755524 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.755525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.755534 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.755611 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.756183 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.756845 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.757644 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.757863 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.757939 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.759196 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.759198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.759198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.759204 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.760731 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.761078 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.761079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.761080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.761137 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.761364 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.761366 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.761366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.761481 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.761585 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.761692 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.761780 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for oryx/127.30.23.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.23.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.23.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.188.keytab.
Entry for principal oryx/127.30.23.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.188.keytab.
I20260809 16:29:56.793260 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.188:46157
--webserver_interface=127.30.23.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.188.keytab
--principal=oryx/127.30.23.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:35975,127.30.23.189:35459,127.30.23.188:46157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:56.881995 9488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:56.882212 9488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:56.882236 9488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:56.883380 9488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:29:56.883424 9488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:56.883440 9488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:56.883451 9488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:29:56.883463 9488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:29:56.884955 9488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:35975,127.30.23.189:35459,127.30.23.188:46157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.188.keytab
--principal=oryx/127.30.23.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.188:46157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.23.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:56.885170 9488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:56.885406 9488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:56.891610 9511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:56.891741 9507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:56.891709 9508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:56.891923 9488 server_base.cc:1034] running on GCE node
I20260809 16:29:56.892241 9488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:56.892551 9488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:56.893776 9488 hybrid_clock.cc:648] HybridClock initialized: now 1786292996893742 us; error 54 us; skew 500 ppm
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:56.896741 9488 init.cc:377] Logged in from keytab as oryx/127.30.23.188@KRBTEST.COM (short username oryx)
I20260809 16:29:56.897226 9488 webserver.cc:460] Webserver started at http://127.30.23.188:46625/ using document root <none> and password file <none>
I20260809 16:29:56.897518 9488 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:56.897586 9488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:56.897745 9488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:29:56.898830 9488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/data/instance:
uuid: "bb08293159084c60b56ecd9abe325431"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "d5a8326545eca6f020c1f264ebc661a9"
server_key_iv: "3a0c4b65ed1d3744f4624e3e43e558a1"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.899227 9488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/wal/instance:
uuid: "bb08293159084c60b56ecd9abe325431"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "d5a8326545eca6f020c1f264ebc661a9"
server_key_iv: "3a0c4b65ed1d3744f4624e3e43e558a1"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.900709 9488 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:29:56.901607 9520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:56.901779 9488 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:56.901899 9488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/wal
uuid: "bb08293159084c60b56ecd9abe325431"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "d5a8326545eca6f020c1f264ebc661a9"
server_key_iv: "3a0c4b65ed1d3744f4624e3e43e558a1"
server_key_version: "encryptionkey@0"
I20260809 16:29:56.901984 9488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:56.923062 9488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:56.925590 9488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:56.925760 9488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:56.930294 9488 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.188:46157
I20260809 16:29:56.930353 9631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.188:46157 every 8 connection(s)
I20260809 16:29:56.930734 9488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/data/info.pb
I20260809 16:29:56.931788 9633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:56.933490 9633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.188@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:29:56.939858 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 9488
I20260809 16:29:56.944053 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.934328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44536 (local address 127.30.23.190:35975)
0809 16:29:56.934410 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.934413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.934515 (+ 102us) server_negotiation.cc:407] Connection header received
0809 16:29:56.935216 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.935218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.935232 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.935293 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.936021 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.936537 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.937647 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.937845 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.937928 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.940746 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.940748 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.940748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.940756 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.942493 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.942918 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.942919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.942920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.942990 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.943309 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.943310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.943311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.943434 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.943490 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.943827 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.943920 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 16:29:56.944216 9635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.934222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:44536)
0809 16:29:56.934397 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.934434 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.935099 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.935406 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.935410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.935668 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.935914 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.935918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.936639 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.936640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.937554 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.937557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.938007 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.938009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.938114 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.938684 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.938699 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.940609 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:56.942584 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.942590 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.942592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.942842 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.943086 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.943087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.943087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.943205 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.943496 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:56.943525 (+ 29us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:56.943677 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 16:29:56.943791 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 16:29:56.943962 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260809 16:29:56.944836 9633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.23.188@KRBTEST.COM
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.188@KRBTEST.COM for oryx/127.30.23.189@KRBTEST.COM
I20260809 16:29:56.949935 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.940459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55988 (local address 127.30.23.188:46157)
0809 16:29:56.940678 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.940684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.940709 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:29:56.940737 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.940739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.940750 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.940835 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.941669 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.942397 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.943295 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.943702 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.943779 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.946555 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.946562 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.946562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.946569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.948487 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.948978 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.948981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.948981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.949062 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.949289 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.949289 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.949290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.949485 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.949642 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.949728 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.949818 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":115,"threads_started":1}
I20260809 16:29:56.952210 9635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.945120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:35459 (local address 127.0.0.1:59386)
0809 16:29:56.945168 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.945178 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.945217 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.945436 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.945437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.945590 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.945660 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.945663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.946494 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.946495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.947036 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.947038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.947429 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.947431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.947513 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.947782 (+ 269us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.947788 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.949407 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:56.951299 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.951300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.951300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.951491 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.951696 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.951697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.951697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.951752 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.951985 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:56.951985 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:56.952026 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 16:29:56.952077 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:29:56.952151 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 16:29:56.952221 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.945216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59386 (local address 127.30.23.189:35459)
0809 16:29:56.945282 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.945284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.945292 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:29:56.945319 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.945320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.945368 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.945416 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.945769 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.946383 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.947100 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.947354 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.947432 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.949478 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.949480 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.949480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.949488 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.951219 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.951572 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.951573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.951573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.951630 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.951825 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.951826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.951826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.951932 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.951984 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.952094 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.952162 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:29:56.952703 9633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:29:56 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292996, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.188@KRBTEST.COM for oryx/127.30.23.188@KRBTEST.COM
I20260809 16:29:56.959054 9635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.952967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:46157 (local address 127.0.0.1:56002)
0809 16:29:56.953034 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.953044 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.953094 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.953180 (+ 86us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.953181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.953297 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.953327 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.953328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.954004 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.954005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.954532 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.954534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.954905 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.954906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.954947 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.955285 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.955290 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.956525 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:56.958080 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.958081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.958081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.958246 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.958452 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.958452 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.958453 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.958506 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.958819 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:56.958819 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:56.958856 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 16:29:56.958920 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 16:29:56.958993 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 16:29:56.959272 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.953021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56002 (local address 127.30.23.188:46157)
0809 16:29:56.953055 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.953057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.953083 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:29:56.953129 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.953129 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.953137 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.953178 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.953443 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.953916 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.954632 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.954832 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.954905 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.956601 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.956602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.956603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.956609 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.957985 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.958314 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.958315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.958315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.958378 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.958591 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.958592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.958592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.958725 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.958791 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.958970 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.959038 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":17}
I20260809 16:29:56.960605 9633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: Bootstrap starting.
I20260809 16:29:56.961530 9633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:56.961831 9633 log.cc:824] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:56.962634 9633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: No bootstrap required, opened a new log
I20260809 16:29:56.964133 9633 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } }
I20260809 16:29:56.964280 9633 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:56.964345 9633 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb08293159084c60b56ecd9abe325431, State: Initialized, Role: FOLLOWER
I20260809 16:29:56.964488 9633 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } }
I20260809 16:29:56.964843 9633 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:56.964811 9644 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } } }
I20260809 16:29:56.964932 9644 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:29:56.966696 9661 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:29:56.966765 9661 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:29:56.969465 9488 master_runner.cc:428] Detected that this master bb08293159084c60b56ecd9abe325431 is joining an existing cluster
I20260809 16:29:56.969530 9488 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.23.188:46157
I20260809 16:29:56.978744 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.970960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:46157 (local address 127.0.0.1:56016)
0809 16:29:56.971187 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.971195 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.971268 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.971421 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.971422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.971709 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.971773 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.971775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.972676 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.972677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.973731 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.973734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.974240 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.974241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.974301 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.974741 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.974746 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.975285 (+ 539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:56.977315 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.977316 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.977317 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.977645 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.977902 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:56.977903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:56.977903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:56.977982 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:56.978309 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:56.978310 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:56.978383 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 16:29:56.978466 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 16:29:56.978620 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":72,"thread_start_us":73,"threads_started":1}
I20260809 16:29:56.978852 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.971107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.30.23.188:46157)
0809 16:29:56.971142 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.971144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.971241 (+ 97us) server_negotiation.cc:407] Connection header received
0809 16:29:56.971331 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.971331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.971339 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.971367 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.971862 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.972584 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.973824 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.974110 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.974153 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.975389 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.975390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.975391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.975397 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:56.977209 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.977748 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.977749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.977749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.977821 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:56.978073 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:56.978074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:56.978074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:56.978201 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:56.978270 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:56.978528 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 16:29:56.978703 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":17}
I20260809 16:29:57.001533 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 8456
I20260809 16:29:57.008757 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.970554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44548 (local address 127.30.23.190:35975)
0809 16:29:56.970602 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.970604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.970996 (+ 392us) server_negotiation.cc:407] Connection header received
0809 16:29:56.971113 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.971114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.971127 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.971161 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.971791 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.972564 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.973815 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.974118 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.974160 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.999103 (+ 24943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.999106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.999107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.999118 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.000917 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.001519 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.001521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.001521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.001587 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.007817 (+ 6230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.007819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.007820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.008049 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.008149 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.008463 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.008598 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:29:57.009802 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.970503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:44548)
0809 16:29:56.970927 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.970937 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.971021 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.971234 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.971235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.971600 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.971693 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.971695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.972666 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.972666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.973710 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.973713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.974252 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.974252 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.974315 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.998231 (+ 23916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.998240 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.998952 (+ 712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:57.001014 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.001015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.001016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.001428 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.007573 (+ 6145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.007575 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.007576 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.007701 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.008199 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:57.008201 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:57.008296 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.009422 (+ 1126us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.009602 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":58,"thread_start_us":79,"threads_started":1}
I20260809 16:29:57.017004 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.970813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59402 (local address 127.30.23.189:35459)
0809 16:29:56.970851 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:56.970853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:56.971133 (+ 280us) server_negotiation.cc:407] Connection header received
0809 16:29:56.971229 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:56.971230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:56.971240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:56.971271 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:56.971791 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.972564 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.973814 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.974118 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.974160 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.999463 (+ 25303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:56.999465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:56.999465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:56.999474 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.001132 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.015876 (+ 14744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.015879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.015880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.016000 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.016285 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.016286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.016286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.016454 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.016542 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.016760 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.016885 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:29:57.017699 9675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:56.970763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:35459 (local address 127.0.0.1:59402)
0809 16:29:56.971076 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:56.971085 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:56.971157 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:56.971349 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:56.971350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:56.971612 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:56.971697 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.971698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.972666 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.972666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.973710 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:56.973713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:56.974245 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:56.974246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:56.974315 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:56.998272 (+ 23957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:56.998277 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:56.998952 (+ 675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:57.001256 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.001257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.001258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.001560 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.016122 (+ 14562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.016123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.016123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.016205 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.016583 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:57.016584 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:57.016639 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.017463 (+ 824us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.017611 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":72,"thread_start_us":79,"threads_started":1}
I20260809 16:29:57.022027 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:33891
--local_ip_for_outbound_sockets=127.30.23.129
--tserver_master_addrs=127.30.23.190:35975
--webserver_port=38995
--webserver_interface=127.30.23.129
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab
--principal=oryx/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.23.190:35975,127.30.23.189:35459
--tserver_master_addrs=127.30.23.190:35975,127.30.23.189:35459,127.30.23.188:46157 with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292995, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.190@KRBTEST.COM for oryx/127.30.23.188@KRBTEST.COM
I20260809 16:29:57.026880 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.019129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56030 (local address 127.30.23.188:46157)
0809 16:29:57.019183 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.019185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.019192 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:29:57.019227 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.019228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.019236 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.019297 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.019702 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.020200 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.020887 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.021073 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.021155 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.023408 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.023410 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.023411 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.023419 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.025640 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.026032 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.026034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.026034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.026139 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.026345 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.026346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.026346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.026466 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.026537 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.026693 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.026785 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:29:57.026881 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.018898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:46157 (local address 127.0.0.1:56030)
0809 16:29:57.018984 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.018996 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.019052 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.019317 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.019318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.019560 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:57.019609 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.019612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.020298 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.020299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.020807 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.020808 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.021168 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.021169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.021236 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.021682 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:57.021688 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:57.023317 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:57.025728 (+ 2411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.025729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.025730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.025929 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.026213 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.026214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.026214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.026282 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.026543 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:57.026544 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:57.026590 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.026667 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.026780 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 16:29:57.027451 8353 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.23.188:46157: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bb08293159084c60b56ecd9abe325431"
member_type: NON_VOTER
last_known_addr {
host: "127.30.23.188"
port: 46157
}
attrs {
promote: true
}
}
dest_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3"
cas_config_opid_index: 5
I20260809 16:29:57.027674 8353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } }
I20260809 16:29:57.028010 9327 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bb08293159084c60b56ecd9abe325431 (127.30.23.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } } }
I20260809 16:29:57.028558 9583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:57.029121 9327 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.23.188:46157
I20260809 16:29:57.029211 9328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } } }
I20260809 16:29:57.029369 9328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:57.029963 9488 master.cc:537] Master@127.30.23.188:46157 shutting down...
I20260809 16:29:57.030645 9583 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 FOLLOWER]: Refusing update from remote peer 1d1d69d1116e40e29f5dfb7ec05c37b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 16:29:57.030951 9328 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 16:29:57.031880 8330 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 -> Peer bb08293159084c60b56ecd9abe325431 (127.30.23.188:46157): Couldn't send request to peer bb08293159084c60b56ecd9abe325431. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 16:29:57.035022 9488 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:57.035125 9488 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:57.035180 9488 tablet_replica.cc:330] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: stopping tablet replica
I20260809 16:29:57.035279 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.028118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:35459 (local address 127.0.0.1:59404)
0809 16:29:57.029672 (+ 1554us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.029684 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.029726 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.031534 (+ 1808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.031536 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.031538 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:57.031584 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.031585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.033146 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.033148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.034059 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.034062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.035011 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.035012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.035082 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.035113 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.035168 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.035215 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1537}
I20260809 16:29:57.035777 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.028248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59404 (local address 127.30.23.189:35459)
0809 16:29:57.028299 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.028302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.029988 (+ 1686us) server_negotiation.cc:407] Connection header received
0809 16:29:57.030020 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.030021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.030047 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.030111 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:57.031648 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.032431 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.034156 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.034786 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.034864 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.034956 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.035670 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.035717 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:29:57.036036 9416 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 LEARNER]: Refusing update from remote peer 1d1d69d1116e40e29f5dfb7ec05c37b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 16:29:57.036378 9328 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 16:29:57.036981 9416 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bb08293159084c60b56ecd9abe325431 (127.30.23.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } } }
I20260809 16:29:57.038082 9462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } } }
I20260809 16:29:57.038177 9462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: This master's current role is: LEARNER
I20260809 16:29:57.038071 9680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d1d69d1116e40e29f5dfb7ec05c37b3. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: NON_VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: true } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } } }
I20260809 16:29:57.038265 9680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: This master's current role is: LEARNER
I20260809 16:29:57.039085 9328 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3: attempting to promote NON_VOTER 8d9c32a955cf44c091f52611fe665054 to VOTER
I20260809 16:29:57.039333 9328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } }
I20260809 16:29:57.040017 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.039883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:46157 (local address 127.0.0.1:56036)
0809 16:29:57.039934 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.039984 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 16:29:57.040067 9416 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 LEARNER]: Refusing update from remote peer 1d1d69d1116e40e29f5dfb7ec05c37b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 16:29:57.040352 9327 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 16:29:57.040396 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.040312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:46157 (local address 127.0.0.1:56042)
0809 16:29:57.040351 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.040377 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 16:29:57.040474 8330 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 -> Peer bb08293159084c60b56ecd9abe325431 (127.30.23.188:46157): Couldn't send request to peer bb08293159084c60b56ecd9abe325431. Status: Network error: Client connection negotiation failed: client connection to 127.30.23.188:46157: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 16:29:57.040973 9682 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8d9c32a955cf44c091f52611fe665054 (127.30.23.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } } }
I20260809 16:29:57.041131 9416 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8d9c32a955cf44c091f52611fe665054 (127.30.23.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } } }
I20260809 16:29:57.042322 9680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } } }
I20260809 16:29:57.042474 9680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:29:57.042699 9327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } } }
I20260809 16:29:57.042783 9327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:57.042852 9328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } } }
I20260809 16:29:57.042920 9328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:57.050371 9488 master.cc:552] Master@127.30.23.188:46157 shutdown complete.
I20260809 16:29:57.050452 9488 master_runner.cc:305] Clearing existing system tablet
I20260809 16:29:57.050592 9488 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:57.050776 9488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:57.051524 9488 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260809 16:29:57.052074 9685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:57.052270 9488 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:57.052340 9488 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:57.052374 9488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/wal
uuid: "bb08293159084c60b56ecd9abe325431"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "d5a8326545eca6f020c1f264ebc661a9"
server_key_iv: "3a0c4b65ed1d3744f4624e3e43e558a1"
server_key_version: "encryptionkey@0"
I20260809 16:29:57.053071 9488 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:57.055105 9488 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 16:29:57.055207 9488 log.cc:1196] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 16:29:57.055438 9488 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: Deleting consensus metadata
I20260809 16:29:57.055584 9488 master_runner.cc:315] Copying system tablet from 127.30.23.190:35975
I20260809 16:29:57.056335 9488 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: tablet copy: Beginning tablet copy session from remote peer at address 127.30.23.190:35975
I20260809 16:29:57.062717 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.056728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.30.23.190:35975)
0809 16:29:57.056782 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.056784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.056906 (+ 122us) server_negotiation.cc:407] Connection header received
0809 16:29:57.056977 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.056978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.056992 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.057049 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.057405 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.057915 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.058681 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.058895 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.058967 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.059948 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.059950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.059950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.059959 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.061475 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.061848 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.061849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.061849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.061910 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.062112 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.062113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.062113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.062231 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.062303 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.062475 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.062595 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:29:57.062747 9695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.056669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:44558)
0809 16:29:57.056828 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.056844 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.056920 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.057054 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.057055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.057240 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:57.057281 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.057283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.058003 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.058005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.058595 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.058598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.058966 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.058967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.059030 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.059528 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:57.059534 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:57.059854 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:57.061570 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.061571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.061571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.061758 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.061985 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.061986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.061986 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.062051 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.062313 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:57.062314 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:57.062360 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.062433 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.062607 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":74,"threads_started":1}
I20260809 16:29:57.063331 8390 tablet_copy_service.cc:140] P 1d1d69d1116e40e29f5dfb7ec05c37b3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bb08293159084c60b56ecd9abe325431 ({username='oryx', principal='oryx/127.30.23.188@KRBTEST.COM'} at 127.0.0.1:44558)
I20260809 16:29:57.063410 8390 tablet_copy_service.cc:161] P 1d1d69d1116e40e29f5dfb7ec05c37b3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bb08293159084c60b56ecd9abe325431 at {username='oryx', principal='oryx/127.30.23.188@KRBTEST.COM'} at 127.0.0.1:44558: session id = bb08293159084c60b56ecd9abe325431-00000000000000000000000000000000
I20260809 16:29:57.063905 8390 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 16:29:57.064388 9488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 16:29:57.065992 9488 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: tablet copy: Starting download of 0 data blocks...
I20260809 16:29:57.066152 9488 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: tablet copy: Starting download of 1 WAL segments...
I20260809 16:29:57.066849 9488 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 16:29:57.067605 8390 tablet_copy_service.cc:342] P 1d1d69d1116e40e29f5dfb7ec05c37b3: Request end of tablet copy session bb08293159084c60b56ecd9abe325431-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.23.188@KRBTEST.COM'} at 127.0.0.1:44558
I20260809 16:29:57.067679 8390 tablet_copy_service.cc:433] P 1d1d69d1116e40e29f5dfb7ec05c37b3: ending tablet copy session bb08293159084c60b56ecd9abe325431-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bb08293159084c60b56ecd9abe325431
I20260809 16:29:57.070550 9488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:57.071861 9698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:57.072012 9488 server_base.cc:1034] running on GCE node
W20260809 16:29:57.071905 9704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:57.072209 9699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:57.072394 9488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:57.072575 9488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:57.073720 9488 hybrid_clock.cc:648] HybridClock initialized: now 1786292997073704 us; error 33 us; skew 500 ppm
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292997, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:57.075273 9488 init.cc:377] Logged in from keytab as oryx/127.30.23.188@KRBTEST.COM (short username oryx)
I20260809 16:29:57.075596 9488 webserver.cc:460] Webserver started at http://127.30.23.188:45535/ using document root <none> and password file <none>
I20260809 16:29:57.075736 9488 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:57.075789 9488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:57.076495 9488 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:29:57.077092 9712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:57.077252 9488 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:29:57.077309 9488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/wal
uuid: "bb08293159084c60b56ecd9abe325431"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "d5a8326545eca6f020c1f264ebc661a9"
server_key_iv: "3a0c4b65ed1d3744f4624e3e43e558a1"
server_key_version: "encryptionkey@0"
I20260809 16:29:57.077402 9488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:57.091223 9488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:57.093117 9488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:57.096637 9488 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.188:46157
I20260809 16:29:57.097012 9488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/master-2/data/info.pb
I20260809 16:29:57.097841 9806 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:29:57.098377 9806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: Bootstrap starting.
I20260809 16:29:57.101200 9806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:29:57.101557 9806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: Bootstrap complete.
I20260809 16:29:57.101831 9806 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } }
I20260809 16:29:57.101931 9806 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bb08293159084c60b56ecd9abe325431, State: Initialized, Role: LEARNER
I20260809 16:29:57.101989 9806 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } }
I20260809 16:29:57.102200 9806 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:29:57.102192 9812 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } } }
I20260809 16:29:57.102491 9812 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: This master's current role is: LEARNER
I20260809 16:29:57.104086 9830 catalog_manager.cc:1259] Loaded cluster ID: b0b8579f5eec4a569d041b1164b4e8b0
I20260809 16:29:57.104182 9830 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: loading cluster ID for follower catalog manager: success
I20260809 16:29:57.105515 9830 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: acquiring CA information for follower catalog manager: success
I20260809 16:29:57.105816 9830 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:29:57.109352 9805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.188:46157 every 8 connection(s)
W20260809 16:29:57.150873 9678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:57.151203 9678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:57.151252 9678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:57.153060 9678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:57.153126 9678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:57.153167 9678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:29:57.155131 9678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab
--principal=oryx/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:33891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=38995
--tserver_master_addrs=127.30.23.190:35975,127.30.23.189:35459,127.30.23.188:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:57.155424 9678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:57.155702 9678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:57.162079 9836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:57.162233 9835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:57.162076 9842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:57.163115 9678 server_base.cc:1034] running on GCE node
I20260809 16:29:57.163451 9678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:57.163800 9678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:57.165010 9678 hybrid_clock.cc:648] HybridClock initialized: now 1786292997164992 us; error 46 us; skew 500 ppm
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292997, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:57.168170 9678 init.cc:377] Logged in from keytab as oryx/127.30.23.129@KRBTEST.COM (short username oryx)
I20260809 16:29:57.168743 9678 webserver.cc:460] Webserver started at http://127.30.23.129:38995/ using document root <none> and password file <none>
I20260809 16:29:57.169008 9678 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:57.169111 9678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:57.170835 9678 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:57.171543 9858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:57.171772 9678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:29:57.171938 9678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "3e682250219344f09ef1b9bd92b8a2f2"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "c43af381a2a0c806b281cae9d15ca261"
server_key_iv: "4b7bd7dd33702d741c7fd1e1f086df0c"
server_key_version: "encryptionkey@0"
I20260809 16:29:57.172330 9678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:57.186165 9678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:57.188572 9678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:57.188700 9678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:57.188920 9678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:57.189251 9678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:57.189306 9678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:57.189378 9678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:57.189410 9678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:57.195902 9678 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:33891
I20260809 16:29:57.195981 10063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:33891 every 8 connection(s)
I20260809 16:29:57.196343 9678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:29:57.200789 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 9678
I20260809 16:29:57.200976 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 8679
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292997, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.188@KRBTEST.COM
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292997, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.189@KRBTEST.COM
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292997, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:29:57.215233 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.197291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:37693 (local address 127.30.23.190:35975)
0809 16:29:57.197371 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.197373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.197553 (+ 180us) server_negotiation.cc:407] Connection header received
0809 16:29:57.198160 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.198162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.198176 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.198210 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.199077 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.199839 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.200882 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.201204 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.201247 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.209117 (+ 7870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.209120 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.209120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.209129 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.211232 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.213932 (+ 2700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.213934 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.213934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.214010 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.214371 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.214372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.214373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.214513 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.214572 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.215016 (+ 444us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.215127 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:29:57.215745 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.197119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:39015 (local address 127.30.23.188:46157)
0809 16:29:57.197266 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.197268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.197666 (+ 398us) server_negotiation.cc:407] Connection header received
0809 16:29:57.198203 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.198205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.198219 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.198284 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.199077 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.200059 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.201178 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.201520 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.201588 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.211322 (+ 9734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.211323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.211324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.211332 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.213067 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.213958 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.213959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.213960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.214022 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.214360 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.214361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.214362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.214518 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.214572 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.215600 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.215674 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260809 16:29:57.216063 10075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.197105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:46157 (local address 127.30.23.129:39015)
0809 16:29:57.197419 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.197467 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.198040 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.198332 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.198333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.198821 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:57.198965 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.198971 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.200138 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.200139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.201096 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.201099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.201646 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.201647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.201762 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.204579 (+ 2817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:57.204583 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:57.209260 (+ 4677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:57.213164 (+ 3904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.213177 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.213180 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.213534 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.214095 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.214095 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.214096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.214280 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.214611 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:57.214615 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:57.214828 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.214965 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.215152 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":15,"thread_start_us":92,"threads_started":1}
I20260809 16:29:57.216493 10074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.197106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.129:37693)
0809 16:29:57.197433 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.197467 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.198037 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.198315 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.198316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.198775 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:57.198965 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.198971 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.199929 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.199930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.200775 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.200778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.201362 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.201363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.201482 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.204378 (+ 2896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:57.204383 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:57.208984 (+ 4601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:57.213626 (+ 4642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.213627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.213627 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.213866 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.214095 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.214095 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.214096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.214280 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.214607 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:57.214615 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:57.214828 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.215391 (+ 563us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.215498 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":12,"spinlock_wait_cycles":1792,"thread_start_us":110,"threads_started":1}
I20260809 16:29:57.217027 10064 heartbeater.cc:347] Connected to a master server at 127.30.23.188:46157
I20260809 16:29:57.217168 10064 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:57.217322 10065 heartbeater.cc:347] Connected to a master server at 127.30.23.190:35975
I20260809 16:29:57.217429 10065 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:57.217507 10064 heartbeater.cc:511] Master 127.30.23.188:46157 requested a full tablet report, sending...
I20260809 16:29:57.217518 10065 heartbeater.cc:511] Master 127.30.23.190:35975 requested a full tablet report, sending...
I20260809 16:29:57.218043 8353 ts_manager.cc:194] Re-registered known tserver with Master: 3e682250219344f09ef1b9bd92b8a2f2 (127.30.23.129:33891)
I20260809 16:29:57.218526 8353 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:37693
I20260809 16:29:57.224411 9747 ts_manager.cc:194] Registered new tserver with Master: 3e682250219344f09ef1b9bd92b8a2f2 (127.30.23.129:33891)
I20260809 16:29:57.226269 10076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.197453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:35459 (local address 127.30.23.129:58647)
0809 16:29:57.197565 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.197575 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.198054 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.198309 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.198313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.198759 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:57.198964 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.198968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.199697 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.199698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.200794 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.200797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.201319 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.201319 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.201479 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.204273 (+ 2794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:57.204282 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:57.209821 (+ 5539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:57.224925 (+ 15104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.224927 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.224928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.225172 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.225530 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.225531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.225531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.225598 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.225878 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:57.225879 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:57.225932 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.226021 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.226156 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":66,"threads_started":1}
I20260809 16:29:57.226251 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.197628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:58647 (local address 127.30.23.189:35459)
0809 16:29:57.197676 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.197678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.197686 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:29:57.198152 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.198153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.198167 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.198225 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.199057 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.199563 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.200890 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.201191 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.201233 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.209978 (+ 8745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.209980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.209981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.209992 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.224827 (+ 14835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.225317 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.225319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.225319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.225425 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.225665 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.225667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.225667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.225809 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.225888 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.226060 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.226153 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:29:57.227130 10066 heartbeater.cc:347] Connected to a master server at 127.30.23.189:35459
I20260809 16:29:57.227205 10066 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:57.227319 10066 heartbeater.cc:511] Master 127.30.23.189:35459 requested a full tablet report, sending...
I20260809 16:29:57.227758 9396 ts_manager.cc:194] Registered new tserver with Master: 3e682250219344f09ef1b9bd92b8a2f2 (127.30.23.129:33891)
I20260809 16:29:57.229213 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:45517
--local_ip_for_outbound_sockets=127.30.23.130
--tserver_master_addrs=127.30.23.190:35975
--webserver_port=34871
--webserver_interface=127.30.23.130
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab
--principal=oryx/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.23.190:35975,127.30.23.189:35459
--tserver_master_addrs=127.30.23.190:35975,127.30.23.189:35459,127.30.23.188:46157 with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:29:57.320465 10078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:57.320704 10078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:57.320731 10078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:57.322077 10078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:57.322129 10078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:57.322160 10078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:29:57.323731 10078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab
--principal=oryx/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:45517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=34871
--tserver_master_addrs=127.30.23.190:35975,127.30.23.189:35459,127.30.23.188:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:57.323987 10078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:57.324273 10078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:29:57.330426 10093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:57.330474 10085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:57.330454 10088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:57.331354 10078 server_base.cc:1034] running on GCE node
I20260809 16:29:57.331709 10078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:57.331967 10078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:57.333173 10078 hybrid_clock.cc:648] HybridClock initialized: now 1786292997333159 us; error 27 us; skew 500 ppm
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292997, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:57.335879 10078 init.cc:377] Logged in from keytab as oryx/127.30.23.130@KRBTEST.COM (short username oryx)
I20260809 16:29:57.336378 10078 webserver.cc:460] Webserver started at http://127.30.23.130:34871/ using document root <none> and password file <none>
I20260809 16:29:57.336618 10078 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:57.336704 10078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:57.338482 10078 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:57.339231 10102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:57.339459 10078 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:29:57.339576 10078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "110100e55f1e404bba5a298ef2699738"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "296b28c6d9d24665e800017243f06a72"
server_key_iv: "59b17622dfa46932f385ac2ea50ff7ce"
server_key_version: "encryptionkey@0"
I20260809 16:29:57.339970 10078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:57.348312 10078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:57.350629 10078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:57.350803 10078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:57.351128 10078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:57.351583 10078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:57.351644 10078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:57.351687 10078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:57.351715 10078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:57.359092 10078 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:45517
I20260809 16:29:57.359150 10278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:45517 every 8 connection(s)
I20260809 16:29:57.359731 10078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:29:57.365612 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 10078
I20260809 16:29:57.365805 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 8910
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292997, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.188@KRBTEST.COM
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292997, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.189@KRBTEST.COM
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292997, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:29:57.395272 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.360555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:44557 (local address 127.30.23.190:35975)
0809 16:29:57.360629 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.360632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.360981 (+ 349us) server_negotiation.cc:407] Connection header received
0809 16:29:57.362264 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.362266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.362278 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.362326 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.363053 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.363814 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.365043 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.365451 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.365538 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.392205 (+ 26667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.392207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.392208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.392216 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.393890 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.394267 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.394268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.394268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.394318 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.394484 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.394484 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.394485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.394619 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.394656 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.395044 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.395133 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:29:57.395884 10288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.360559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.130:44557)
0809 16:29:57.360778 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.360815 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.362104 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.362379 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.362380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.362807 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:57.362938 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.362940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.363887 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.363888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.364941 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.364944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.365605 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.365606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.365735 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.390265 (+ 24530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:57.390272 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:57.392060 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:57.393994 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.393995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.393996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.394197 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.394379 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.394380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.394380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.394430 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.394693 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:57.394701 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:57.394903 (+ 202us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.395409 (+ 506us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.395605 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":105,"thread_start_us":98,"threads_started":1}
I20260809 16:29:57.396363 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.361180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:49029 (local address 127.30.23.189:35459)
0809 16:29:57.361270 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.361272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.361372 (+ 100us) server_negotiation.cc:407] Connection header received
0809 16:29:57.362243 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.362244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.362259 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.362292 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.363051 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.363809 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.365089 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.365479 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.365542 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.387891 (+ 22349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.387893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.387894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.388289 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.390325 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.393901 (+ 3576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.393904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.393904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.393993 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.394909 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.394910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.394911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.395119 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.395192 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.396045 (+ 853us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.396178 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 16:29:57.396425 10280 heartbeater.cc:347] Connected to a master server at 127.30.23.190:35975
I20260809 16:29:57.396543 10280 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:57.396735 10280 heartbeater.cc:511] Master 127.30.23.190:35975 requested a full tablet report, sending...
I20260809 16:29:57.397107 8353 ts_manager.cc:194] Re-registered known tserver with Master: 110100e55f1e404bba5a298ef2699738 (127.30.23.130:45517)
I20260809 16:29:57.397166 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.360681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:48609 (local address 127.30.23.188:46157)
0809 16:29:57.360727 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.360729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.360902 (+ 173us) server_negotiation.cc:407] Connection header received
0809 16:29:57.362191 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.362192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.362209 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.362275 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.363031 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.363631 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.365132 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.365426 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.365498 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.390939 (+ 25441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.390941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.390941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.390952 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.393272 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.394499 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.394501 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.394501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.394584 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.395464 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.395465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.395466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.395668 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.395741 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.396855 (+ 1114us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.396995 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:29:57.397603 8353 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:44557
I20260809 16:29:57.397747 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.360559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:46157 (local address 127.30.23.130:48609)
0809 16:29:57.360763 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.360816 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.362073 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.362348 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.362351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.362686 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:57.362930 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.362934 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.363761 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.363762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.365015 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.365018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.365573 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.365574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.365735 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.383625 (+ 17890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:57.383638 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:57.387202 (+ 3564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:57.393442 (+ 6240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.393444 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.393445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.393763 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.394688 (+ 925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.394689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.394690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.394795 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.395788 (+ 993us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:57.395789 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:57.395859 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.397398 (+ 1539us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.397547 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":72,"thread_start_us":115,"threads_started":1}
I20260809 16:29:57.398041 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.360895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:35459 (local address 127.30.23.130:49029)
0809 16:29:57.361207 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.361219 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.362104 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.362368 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.362369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.362807 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:57.362938 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.362940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.364094 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.364094 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.365015 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.365018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.383156 (+ 18138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.383158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.383287 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.384445 (+ 1158us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:57.384451 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:57.387734 (+ 3283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:57.390441 (+ 2707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.390452 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.390455 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.390811 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.394102 (+ 3291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.394103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.394104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.394376 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.395253 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:57.395254 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:57.395320 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.396469 (+ 1149us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.396643 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":8,"thread_start_us":60,"threads_started":1}
I20260809 16:29:57.398747 10279 heartbeater.cc:347] Connected to a master server at 127.30.23.188:46157
I20260809 16:29:57.398793 10279 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:57.398861 10279 heartbeater.cc:511] Master 127.30.23.188:46157 requested a full tablet report, sending...
I20260809 16:29:57.399135 9747 ts_manager.cc:194] Registered new tserver with Master: 110100e55f1e404bba5a298ef2699738 (127.30.23.130:45517)
I20260809 16:29:57.400923 10283 heartbeater.cc:347] Connected to a master server at 127.30.23.189:35459
I20260809 16:29:57.400976 10283 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:57.401023 10283 heartbeater.cc:511] Master 127.30.23.189:35459 requested a full tablet report, sending...
I20260809 16:29:57.401328 9396 ts_manager.cc:194] Registered new tserver with Master: 110100e55f1e404bba5a298ef2699738 (127.30.23.130:45517)
I20260809 16:29:57.406105 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:35311
--local_ip_for_outbound_sockets=127.30.23.131
--tserver_master_addrs=127.30.23.190:35975
--webserver_port=43795
--webserver_interface=127.30.23.131
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab
--principal=oryx/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.23.190:35975,127.30.23.189:35459
--tserver_master_addrs=127.30.23.190:35975,127.30.23.189:35459,127.30.23.188:46157 with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:29:57.445838 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.442585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:46157 (local address 127.0.0.1:56048)
0809 16:29:57.442713 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.442726 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.442787 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.442988 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.442990 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.442992 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:57.443043 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.443046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.444092 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.444093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.444824 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.444826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.445643 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.445644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.445697 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.445700 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.445732 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.445773 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 16:29:57.445858 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.442676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.30.23.188:46157)
0809 16:29:57.442745 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.442747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.442779 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:29:57.442840 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.442841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.442866 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.442957 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:57.443167 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.443986 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.444927 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.445561 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.445639 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.445714 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.445747 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.445784 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:29:57.446218 9762 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 16:29:57.446390 9812 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d1d69d1116e40e29f5dfb7ec05c37b3. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } } }
I20260809 16:29:57.446529 9812 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: This master's current role is: LEARNER
W20260809 16:29:57.491791 10290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:29:57.492074 10290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:29:57.492122 10290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:29:57.493713 10290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:29:57.493786 10290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:29:57.493851 10290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:29:57.495667 10290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab
--principal=oryx/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:35311
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=43795
--tserver_master_addrs=127.30.23.190:35975,127.30.23.189:35459,127.30.23.188:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:29:57.495970 10290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:29:57.496238 10290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:29:57.502046 10290 server_base.cc:1034] running on GCE node
W20260809 16:29:57.501936 10304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:57.501969 10299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:29:57.501964 10297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:29:57.502499 10290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:29:57.502810 10290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:29:57.503998 10290 hybrid_clock.cc:648] HybridClock initialized: now 1786292997503986 us; error 34 us; skew 500 ppm
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292997, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:29:57.507004 10290 init.cc:377] Logged in from keytab as oryx/127.30.23.131@KRBTEST.COM (short username oryx)
I20260809 16:29:57.507532 10290 webserver.cc:460] Webserver started at http://127.30.23.131:43795/ using document root <none> and password file <none>
I20260809 16:29:57.507822 10290 fs_manager.cc:359] Metadata directory not provided
I20260809 16:29:57.507922 10290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:29:57.509617 10290 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:29:57.510207 10317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:57.510408 10290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:29:57.510515 10290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "0a81ae62aedb4143a7ecd70c9e9742fb"
format_stamp: "Formatted at 2026-08-09 16:29:56 on dist-test-slave-j2vl"
server_key: "a0fad6d05c29f3893fcf219b8102e46e"
server_key_iv: "7c11882dacc8daff66c376e1befefcfc"
server_key_version: "encryptionkey@0"
I20260809 16:29:57.510878 10290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:29:57.535475 10290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:29:57.537981 10290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:29:57.538151 10290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:29:57.538482 10290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:29:57.538861 10290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:29:57.538905 10290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:57.538932 10290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:29:57.538947 10290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:29:57.546691 10290 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:35311
I20260809 16:29:57.546731 10527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:35311 every 8 connection(s)
I20260809 16:29:57.547144 10290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:29:57.551908 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 10290
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292997, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.189@KRBTEST.COM
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292997, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:29:57.560498 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.547956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:36547 (local address 127.30.23.190:35975)
0809 16:29:57.548012 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.548015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.548616 (+ 601us) server_negotiation.cc:407] Connection header received
0809 16:29:57.549081 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.549082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.549098 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.549136 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.550015 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.550769 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.551753 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.552062 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.552113 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.556408 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.556410 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.556410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.556418 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.559076 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.559511 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.559512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.559512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.559575 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.559781 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.559782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.559782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.559897 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.559939 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.560284 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.560387 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:29:57.560904 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.548374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.131:36547)
0809 16:29:57.548529 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.548540 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.548959 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.549217 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.549222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.549845 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:57.549901 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.549903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.550861 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.550862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.551619 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.551621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.552225 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.552226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.552367 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.553309 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:57.553315 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:57.556301 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:57.559180 (+ 2879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.559180 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.559181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.559396 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.559654 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.559655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.559655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.559713 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.559987 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:57.559988 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:57.560138 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.560638 (+ 500us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.560750 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":24,"thread_start_us":72,"threads_started":1}
I20260809 16:29:57.560921 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.548269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:35459 (local address 127.30.23.131:41489)
0809 16:29:57.548384 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.548395 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.548947 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.549215 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.549219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.549656 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:57.549876 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.549881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.550848 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.550849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.551871 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.551872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.552472 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.552472 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.552553 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.553254 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:57.553267 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:57.556296 (+ 3029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:57.558766 (+ 2470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.558775 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.558778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.559217 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.559465 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.559465 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.559466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.559606 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.559960 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:57.559967 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:57.560140 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.560425 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.560620 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":43,"thread_start_us":67,"threads_started":1}
I20260809 16:29:57.561407 10530 heartbeater.cc:347] Connected to a master server at 127.30.23.190:35975
I20260809 16:29:57.561522 10530 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:57.561730 10530 heartbeater.cc:511] Master 127.30.23.190:35975 requested a full tablet report, sending...
Aug 09 16:29:57 dist-test-slave-j2vl krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786292997, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.188@KRBTEST.COM
I20260809 16:29:57.562110 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.548368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:41489 (local address 127.30.23.189:35459)
0809 16:29:57.548417 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.548419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.548458 (+ 39us) server_negotiation.cc:407] Connection header received
0809 16:29:57.549060 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.549062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.549076 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.549138 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.549987 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.550752 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.551986 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.552321 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.552369 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.556417 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.556418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.556419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.556427 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.558663 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.559319 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.559321 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.559321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.559381 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.559692 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.559693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.559693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.559839 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.559903 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.560308 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.562004 (+ 1696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:29:57.562259 8353 ts_manager.cc:194] Re-registered known tserver with Master: 0a81ae62aedb4143a7ecd70c9e9742fb (127.30.23.131:35311)
I20260809 16:29:57.562489 10534 heartbeater.cc:347] Connected to a master server at 127.30.23.189:35459
I20260809 16:29:57.562536 10534 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:57.562788 8353 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:36547
I20260809 16:29:57.562844 10534 heartbeater.cc:511] Master 127.30.23.189:35459 requested a full tablet report, sending...
I20260809 16:29:57.563159 9396 ts_manager.cc:194] Registered new tserver with Master: 0a81ae62aedb4143a7ecd70c9e9742fb (127.30.23.131:35311)
I20260809 16:29:57.565269 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.553309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59410 (local address 127.30.23.189:35459)
0809 16:29:57.553527 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.553529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.553738 (+ 209us) server_negotiation.cc:407] Connection header received
0809 16:29:57.553848 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.553849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.553864 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.553911 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.554686 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.555235 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.556530 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.556874 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.556924 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.559045 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.559046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.559047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.559054 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.561013 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.564191 (+ 3178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.564193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.564193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.564286 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.564577 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.564578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.564578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.564734 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.564846 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.565099 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.565198 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":91,"threads_started":1}
I20260809 16:29:57.566121 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.548028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:44593 (local address 127.30.23.188:46157)
0809 16:29:57.548076 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.548078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.548346 (+ 268us) server_negotiation.cc:407] Connection header received
0809 16:29:57.549082 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.549083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.549098 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.549136 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.550002 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.550555 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.551986 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.552328 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.552380 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.562361 (+ 9981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.562362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.562363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.562371 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.564456 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.565178 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.565179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.565180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.565251 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.565527 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.565529 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.565529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.565685 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.565730 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.565938 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.566024 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:29:57.566150 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.547995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:46157 (local address 127.30.23.131:44593)
0809 16:29:57.548195 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.548235 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.548947 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.549217 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.549222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.549673 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:29:57.549879 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.549882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.550665 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.550666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.551868 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.551871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.552470 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.552471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.552553 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.553476 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:29:57.553481 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:29:57.562223 (+ 8742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:29:57.564781 (+ 2558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.564782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.564783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.565075 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.565338 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:29:57.565361 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:29:57.565362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:29:57.565436 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:29:57.565783 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:29:57.565784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:29:57.565828 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.565894 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.566025 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":32,"thread_start_us":104,"threads_started":1}
I20260809 16:29:57.566425 10529 heartbeater.cc:347] Connected to a master server at 127.30.23.188:46157
I20260809 16:29:57.566507 10529 heartbeater.cc:464] Registering TS with master...
I20260809 16:29:57.566592 10529 heartbeater.cc:511] Master 127.30.23.188:46157 requested a full tablet report, sending...
I20260809 16:29:57.566872 9747 ts_manager.cc:194] Registered new tserver with Master: 0a81ae62aedb4143a7ecd70c9e9742fb (127.30.23.131:35311)
I20260809 16:29:57.567999 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.553048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44564 (local address 127.30.23.190:35975)
0809 16:29:57.553467 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.553469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.557396 (+ 3927us) server_negotiation.cc:407] Connection header received
0809 16:29:57.557545 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.557546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.557562 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.557651 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.559579 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.560340 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.561652 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.562001 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.562076 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.564264 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.564266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.564266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.564274 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.566113 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.567114 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.567115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.567116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.567181 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.567411 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.567412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.567412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.567552 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.567659 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.567808 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.567903 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":97,"threads_started":1}
I20260809 16:29:57.568563 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.553783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56054 (local address 127.30.23.188:46157)
0809 16:29:57.554023 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.554025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.554212 (+ 187us) server_negotiation.cc:407] Connection header received
0809 16:29:57.554238 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.554239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.554252 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.554325 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.555155 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.555685 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.556776 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.557130 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.557193 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.559281 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.559282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.559283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.559290 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.567205 (+ 7915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.567633 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.567635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.567635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.567701 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.567919 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.567920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.567921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.568073 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.568183 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.568379 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.568474 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":59,"threads_started":1}
I20260809 16:29:57.569777 8353 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:29:57.571514 8353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:29:57.572353 9328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: NON_VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: true } } }
I20260809 16:29:57.572453 9328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:57.573565 9328 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3: attempting to promote NON_VOTER bb08293159084c60b56ecd9abe325431 to VOTER
I20260809 16:29:57.573694 9328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } }
I20260809 16:29:57.574115 9762 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 LEARNER]: Refusing update from remote peer 1d1d69d1116e40e29f5dfb7ec05c37b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 16:29:57.574278 10564 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 16:29:57.574409 9416 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Refusing update from remote peer 1d1d69d1116e40e29f5dfb7ec05c37b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 16:29:57.574627 10564 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 16:29:57.575153 10567 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, bb08293159084c60b56ecd9abe325431 (127.30.23.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } } }
I20260809 16:29:57.575294 9762 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, bb08293159084c60b56ecd9abe325431 (127.30.23.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } } }
I20260809 16:29:57.576231 10567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } } }
I20260809 16:29:57.576313 10567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:57.576323 9812 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } } }
I20260809 16:29:57.576385 10567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } } }
I20260809 16:29:57.576424 9812 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:29:57.576433 10567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:57.576776 9328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } } }
I20260809 16:29:57.576874 9328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:29:57.576898 9416 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, bb08293159084c60b56ecd9abe325431 (127.30.23.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } } }
I20260809 16:29:57.578115 10600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } } }
I20260809 16:29:57.578230 10600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:29:57.581668 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.577716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37760 (local address 127.30.23.130:45517)
0809 16:29:57.577902 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.577908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.577950 (+ 42us) server_negotiation.cc:407] Connection header received
0809 16:29:57.578030 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.578032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.578051 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.578127 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:57.578387 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.579538 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.580563 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.581235 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.581333 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.581473 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.581551 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.581590 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":87,"threads_started":1}
I20260809 16:29:57.581658 10598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.577472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:45517 (local address 127.0.0.1:37760)
0809 16:29:57.577889 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.577901 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.577963 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.578202 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.578203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.578205 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:57.578275 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.578277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.579638 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.579639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.580477 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.580479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.581394 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.581395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.581468 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.581473 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.581536 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.581579 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":23,"thread_start_us":89,"threads_started":1}
I20260809 16:29:57.582070 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.577711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:35311 (local address 127.0.0.1:57152)
0809 16:29:57.578331 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.578351 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.578417 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.578612 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.578613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.578615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:57.578654 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.578656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.579880 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.579881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.580929 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.580931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.581850 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.581851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.581909 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.581914 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.581944 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.581991 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":94,"threads_started":1}
I20260809 16:29:57.582168 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.577342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:33891 (local address 127.0.0.1:39362)
0809 16:29:57.577487 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.577499 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.577547 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.578707 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.578708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.578709 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:57.578751 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.578753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.579575 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.579576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.580723 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.580726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.581895 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.581896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.581958 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.581962 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.582022 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.582058 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":39}
I20260809 16:29:57.582530 10586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.577475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39362 (local address 127.30.23.129:33891)
0809 16:29:57.578558 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.578562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.578572 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:29:57.578611 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.578614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.578629 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.578710 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:57.578814 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.579489 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.580818 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.581716 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.581877 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.581965 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.582235 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.582280 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1041,"thread_start_us":104,"threads_started":1}
I20260809 16:29:57.582700 10185 tablet_service.cc:1467] Processing CreateTablet for tablet 41bf58f2d1ca444d942e625adb082644 (DEFAULT_TABLE table=test-table [id=15618f306c0b4bec8b9669ac3f0af232]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:57.583099 10185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41bf58f2d1ca444d942e625adb082644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:57.583163 9950 tablet_service.cc:1467] Processing CreateTablet for tablet 41bf58f2d1ca444d942e625adb082644 (DEFAULT_TABLE table=test-table [id=15618f306c0b4bec8b9669ac3f0af232]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:57.583537 9950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41bf58f2d1ca444d942e625adb082644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:57.585749 10593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.577636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57152 (local address 127.30.23.131:35311)
0809 16:29:57.577930 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.577935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.578414 (+ 479us) server_negotiation.cc:407] Connection header received
0809 16:29:57.578488 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.578490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.578507 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.578725 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:57.578779 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.579804 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.581016 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.581743 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.585482 (+ 3739us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.585578 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.585608 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.585656 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":76,"threads_started":1}
I20260809 16:29:57.586375 10401 tablet_service.cc:1467] Processing CreateTablet for tablet 41bf58f2d1ca444d942e625adb082644 (DEFAULT_TABLE table=test-table [id=15618f306c0b4bec8b9669ac3f0af232]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:29:57.586647 10602 tablet_bootstrap.cc:492] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2: Bootstrap starting.
I20260809 16:29:57.586710 10401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41bf58f2d1ca444d942e625adb082644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:29:57.587363 10602 tablet_bootstrap.cc:654] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:57.587656 10602 log.cc:824] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:57.588451 10602 tablet_bootstrap.cc:492] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2: No bootstrap required, opened a new log
I20260809 16:29:57.588506 10602 ts_tablet_manager.cc:1397] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:29:57.589919 10602 raft_consensus.cc:348] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } }
I20260809 16:29:57.590073 10602 raft_consensus.cc:374] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:57.590118 10602 raft_consensus.cc:729] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e682250219344f09ef1b9bd92b8a2f2, State: Initialized, Role: FOLLOWER
I20260809 16:29:57.590073 10604 tablet_bootstrap.cc:492] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb: Bootstrap starting.
I20260809 16:29:57.590229 10602 consensus_queue.cc:260] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } }
I20260809 16:29:57.590550 10602 ts_tablet_manager.cc:1428] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:29:57.590708 10065 heartbeater.cc:503] Master 127.30.23.190:35975 was elected leader, sending a full tablet report...
I20260809 16:29:57.591470 10604 tablet_bootstrap.cc:654] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:57.592088 10604 log.cc:824] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:57.592393 10610 tablet_bootstrap.cc:492] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738: Bootstrap starting.
I20260809 16:29:57.592772 10604 tablet_bootstrap.cc:492] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb: No bootstrap required, opened a new log
I20260809 16:29:57.592857 10604 ts_tablet_manager.cc:1397] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:29:57.594076 10610 tablet_bootstrap.cc:654] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738: Neither blocks nor log segments found. Creating new log.
I20260809 16:29:57.594427 10610 log.cc:824] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738: Log is configured to *not* fsync() on all Append() calls
I20260809 16:29:57.595301 10610 tablet_bootstrap.cc:492] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738: No bootstrap required, opened a new log
I20260809 16:29:57.595382 10610 ts_tablet_manager.cc:1397] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:29:57.595795 10604 raft_consensus.cc:348] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } }
I20260809 16:29:57.595983 10604 raft_consensus.cc:374] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:57.596020 10604 raft_consensus.cc:729] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a81ae62aedb4143a7ecd70c9e9742fb, State: Initialized, Role: FOLLOWER
I20260809 16:29:57.596189 10604 consensus_queue.cc:260] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } }
I20260809 16:29:57.596483 10604 ts_tablet_manager.cc:1428] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 16:29:57.596518 10530 heartbeater.cc:503] Master 127.30.23.190:35975 was elected leader, sending a full tablet report...
I20260809 16:29:57.598052 10610 raft_consensus.cc:348] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } }
I20260809 16:29:57.598186 10610 raft_consensus.cc:374] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:29:57.598212 10610 raft_consensus.cc:729] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 110100e55f1e404bba5a298ef2699738, State: Initialized, Role: FOLLOWER
I20260809 16:29:57.598347 10610 consensus_queue.cc:260] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } }
I20260809 16:29:57.598614 10610 ts_tablet_manager.cc:1428] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:29:57.598681 10280 heartbeater.cc:503] Master 127.30.23.190:35975 was elected leader, sending a full tablet report...
W20260809 16:29:57.610919 10284 tablet.cc:2369] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:29:57.697358 10070 tablet.cc:2369] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:29:57.798869 10535 tablet.cc:2369] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:29:57.891342 10613 raft_consensus.cc:482] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:29:57.891522 10613 raft_consensus.cc:504] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } }
I20260809 16:29:57.891894 10613 leader_election.cc:288] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e682250219344f09ef1b9bd92b8a2f2 (127.30.23.129:33891), 110100e55f1e404bba5a298ef2699738 (127.30.23.130:45517)
I20260809 16:29:57.896090 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.892207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:54141 (local address 127.30.23.130:45517)
0809 16:29:57.892262 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.892264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.892275 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:29:57.892312 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.892313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.892322 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.892404 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:57.892645 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.893471 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.895090 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.895663 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.895753 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.895763 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.895961 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.896020 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:29:57.896093 10586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.892187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:54187 (local address 127.30.23.129:33891)
0809 16:29:57.892239 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.892242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.892377 (+ 135us) server_negotiation.cc:407] Connection header received
0809 16:29:57.892411 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.892413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.892421 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.892482 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:57.892644 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.893443 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.895092 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.895663 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.895753 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.895763 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.895961 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.896013 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:29:57.896092 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.892147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:33891 (local address 127.30.23.131:54187)
0809 16:29:57.892319 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.892326 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.892355 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.892497 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.892498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.892500 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:57.892544 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.892545 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.893576 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.893577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.894970 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.894975 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.895776 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.895777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.895857 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.895861 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.895925 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.895974 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":62}
I20260809 16:29:57.896104 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.892113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:45517 (local address 127.30.23.131:54141)
0809 16:29:57.892199 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:57.892212 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:57.892259 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:57.892423 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:57.892424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:57.892426 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:57.892527 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.892529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.893576 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.893577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.894988 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:57.894990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.895768 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.895770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:57.895857 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.895861 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:29:57.895923 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:29:57.895986 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":103}
I20260809 16:29:57.896732 10211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41bf58f2d1ca444d942e625adb082644" candidate_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110100e55f1e404bba5a298ef2699738" is_pre_election: true
I20260809 16:29:57.896790 9990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41bf58f2d1ca444d942e625adb082644" candidate_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e682250219344f09ef1b9bd92b8a2f2" is_pre_election: true
I20260809 16:29:57.896922 10211 raft_consensus.cc:2454] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a81ae62aedb4143a7ecd70c9e9742fb in term 0.
I20260809 16:29:57.897020 9990 raft_consensus.cc:2454] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a81ae62aedb4143a7ecd70c9e9742fb in term 0.
I20260809 16:29:57.897197 10318 leader_election.cc:302] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a81ae62aedb4143a7ecd70c9e9742fb, 110100e55f1e404bba5a298ef2699738; no voters:
I20260809 16:29:57.897388 10613 raft_consensus.cc:2781] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:29:57.897478 10613 raft_consensus.cc:482] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:29:57.897507 10613 raft_consensus.cc:3037] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:57.898265 10613 raft_consensus.cc:504] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } }
I20260809 16:29:57.898444 10613 leader_election.cc:288] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [CANDIDATE]: Term 1 election: Requested vote from peers 3e682250219344f09ef1b9bd92b8a2f2 (127.30.23.129:33891), 110100e55f1e404bba5a298ef2699738 (127.30.23.130:45517)
I20260809 16:29:57.898602 9990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41bf58f2d1ca444d942e625adb082644" candidate_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e682250219344f09ef1b9bd92b8a2f2"
I20260809 16:29:57.898617 10211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41bf58f2d1ca444d942e625adb082644" candidate_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110100e55f1e404bba5a298ef2699738"
I20260809 16:29:57.898700 10211 raft_consensus.cc:3037] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:57.898705 9990 raft_consensus.cc:3037] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:29:57.899418 10211 raft_consensus.cc:2454] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a81ae62aedb4143a7ecd70c9e9742fb in term 1.
I20260809 16:29:57.899452 9990 raft_consensus.cc:2454] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a81ae62aedb4143a7ecd70c9e9742fb in term 1.
I20260809 16:29:57.899634 10322 leader_election.cc:302] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a81ae62aedb4143a7ecd70c9e9742fb, 3e682250219344f09ef1b9bd92b8a2f2; no voters:
I20260809 16:29:57.899762 10613 raft_consensus.cc:2781] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:29:57.899938 10613 raft_consensus.cc:686] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 1 LEADER]: Becoming Leader. State: Replica: 0a81ae62aedb4143a7ecd70c9e9742fb, State: Running, Role: LEADER
I20260809 16:29:57.900054 10613 consensus_queue.cc:237] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } }
I20260809 16:29:57.901124 8353 catalog_manager.cc:5495] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a81ae62aedb4143a7ecd70c9e9742fb (127.30.23.131). New cstate: current_term: 1 leader_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } health_report { overall_health: UNKNOWN } } }
I20260809 16:29:57.934319 10593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.931777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57166 (local address 127.30.23.131:35311)
0809 16:29:57.931841 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.931843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.931855 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:29:57.931893 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.931894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.931914 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.931996 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:29:57.932154 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.932638 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.933473 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.933682 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.933746 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.933983 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:29:57.934099 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:29:57.934186 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.934222 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.934245 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:29:57.936457 10211 raft_consensus.cc:1261] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738 [term 1 FOLLOWER]: Refusing update from remote peer 0a81ae62aedb4143a7ecd70c9e9742fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:57.936503 9990 raft_consensus.cc:1261] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2 [term 1 FOLLOWER]: Refusing update from remote peer 0a81ae62aedb4143a7ecd70c9e9742fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:29:57.936906 10613 consensus_queue.cc:1035] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:57.937095 10615 consensus_queue.cc:1035] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:29:57.940021 10623 mvcc.cc:205] Tried to move back new op lower bound from 7316656119544356864 to 7316656119399243776. Current Snapshot: MvccSnapshot[applied={T|T < 7316656119544356864}]
I20260809 16:29:57.941599 10619 mvcc.cc:205] Tried to move back new op lower bound from 7316656119544356864 to 7316656119399243776. Current Snapshot: MvccSnapshot[applied={T|T < 7316656119544356864 or (T in {7316656119544356864})}]
I20260809 16:29:57.943076 8353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:57.943183 8353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:29:57.944847 8353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d1d69d1116e40e29f5dfb7ec05c37b3: Sending DeleteTablet for 3 replicas of tablet 41bf58f2d1ca444d942e625adb082644
I20260809 16:29:57.945204 9950 tablet_service.cc:1514] Processing DeleteTablet for tablet 41bf58f2d1ca444d942e625adb082644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:57 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:39362
I20260809 16:29:57.945390 10401 tablet_service.cc:1514] Processing DeleteTablet for tablet 41bf58f2d1ca444d942e625adb082644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:57 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:57152
I20260809 16:29:57.945582 10637 tablet_replica.cc:330] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2: stopping tablet replica
I20260809 16:29:57.945636 10638 tablet_replica.cc:330] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb: stopping tablet replica
I20260809 16:29:57.945715 10637 raft_consensus.cc:2229] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:57.945885 10637 raft_consensus.cc:2258] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:57.945916 10638 raft_consensus.cc:2229] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:29:57.946137 10638 raft_consensus.cc:2258] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:57.946378 10185 tablet_service.cc:1514] Processing DeleteTablet for tablet 41bf58f2d1ca444d942e625adb082644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:29:57 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:37760
I20260809 16:29:57.946558 10644 tablet_replica.cc:330] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738: stopping tablet replica
I20260809 16:29:57.946653 10644 raft_consensus.cc:2229] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:29:57.946751 10644 raft_consensus.cc:2258] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:29:57.947135 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 8302
I20260809 16:29:57.947252 10644 ts_tablet_manager.cc:1905] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:57.958216 10638 ts_tablet_manager.cc:1905] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:57.958248 10644 ts_tablet_manager.cc:1918] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:29:57.960403 10638 ts_tablet_manager.cc:1918] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:29:57.960431 10644 log.cc:1196] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/41bf58f2d1ca444d942e625adb082644
I20260809 16:29:57.960503 10638 log.cc:1196] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/41bf58f2d1ca444d942e625adb082644
I20260809 16:29:57.960798 10644 ts_tablet_manager.cc:1939] T 41bf58f2d1ca444d942e625adb082644 P 110100e55f1e404bba5a298ef2699738: Deleting consensus metadata
I20260809 16:29:57.960809 10638 ts_tablet_manager.cc:1939] T 41bf58f2d1ca444d942e625adb082644 P 0a81ae62aedb4143a7ecd70c9e9742fb: Deleting consensus metadata
W20260809 16:29:57.964409 10109 connection.cc:535] server connection from 127.0.0.1:37760 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 16:29:57.964432 10319 connection.cc:535] server connection from 127.0.0.1:57152 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:29:57.965161 10637 ts_tablet_manager.cc:1905] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:29:57.967084 10637 ts_tablet_manager.cc:1918] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:29:57.967177 10637 log.cc:1196] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-0/wal/wals/41bf58f2d1ca444d942e625adb082644
I20260809 16:29:57.967478 10637 ts_tablet_manager.cc:1939] T 41bf58f2d1ca444d942e625adb082644 P 3e682250219344f09ef1b9bd92b8a2f2: Deleting consensus metadata
W20260809 16:29:57.967952 9861 connection.cc:441] server connection from 127.0.0.1:39362 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39362 (request call id 1) could send its response
I20260809 16:29:57.969241 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.946485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56062 (local address 127.30.23.188:46157)
0809 16:29:57.946541 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.946544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.957935 (+ 11391us) server_negotiation.cc:407] Connection header received
0809 16:29:57.958034 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.958036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.958054 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.958116 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.960407 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.961272 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.962160 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.962408 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.962511 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.963986 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.963988 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.963988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.963997 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.966577 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.967068 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.967070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.967070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.967134 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.968132 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.968133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.968134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.968315 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.968936 (+ 621us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.968989 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.969117 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:29:57.976948 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.968914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59414 (local address 127.30.23.189:35459)
0809 16:29:57.968965 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.968967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.968977 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:29:57.969005 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.969006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.969021 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.969097 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.970268 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.970825 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.971634 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.971927 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.972014 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.973587 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.973589 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.973589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.973598 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.975537 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.975979 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.975981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.975981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.976065 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.976272 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.976273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.976273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.976444 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.976552 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.976704 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.976833 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:29:57.976999 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:57.968976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56074 (local address 127.30.23.188:46157)
0809 16:29:57.969020 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:57.969022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:57.969137 (+ 115us) server_negotiation.cc:407] Connection header received
0809 16:29:57.969241 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:57.969242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:57.969252 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:57.969282 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:29:57.970073 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.970843 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.971976 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:57.972280 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:57.972321 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:57.973929 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:29:57.973931 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:29:57.973932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:29:57.973941 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:29:57.975706 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.976169 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.976171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.976171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.976227 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:29:57.976433 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:29:57.976433 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:29:57.976434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:29:57.976588 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:29:57.976693 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:57.976833 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 16:29:57.976908 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:29:58.941712 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:58.941318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.129:54571)
0809 16:29:58.941531 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:58.941652 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":111,"threads_started":1}
W20260809 16:29:58.941951 10065 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:35975 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
I20260809 16:29:58.942808 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:58.942556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.131:46219)
0809 16:29:58.942681 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:58.942760 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
W20260809 16:29:58.942996 10530 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:35975 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
I20260809 16:29:58.944056 10657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:58.943777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.130:53681)
0809 16:29:58.943926 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:58.944012 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
W20260809 16:29:58.944217 10280 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:35975 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
I20260809 16:29:59.445771 10659 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1d1d69d1116e40e29f5dfb7ec05c37b3)
I20260809 16:29:59.445847 10660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1d1d69d1116e40e29f5dfb7ec05c37b3)
I20260809 16:29:59.446036 10660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } }
I20260809 16:29:59.446058 10659 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } }
I20260809 16:29:59.446331 10660 leader_election.cc:288] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975), bb08293159084c60b56ecd9abe325431 (127.30.23.188:46157)
I20260809 16:29:59.446424 10659 leader_election.cc:288] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975), 8d9c32a955cf44c091f52611fe665054 (127.30.23.189:35459)
I20260809 16:29:59.446768 10662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.446479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45002)
0809 16:29:59.446650 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:59.446714 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":62,"threads_started":1}
W20260809 16:29:59.446956 9364 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
I20260809 16:29:59.447297 10666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.446504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45018)
0809 16:29:59.446750 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:59.446801 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":56,"threads_started":1}
W20260809 16:29:59.447479 9713 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
I20260809 16:29:59.447662 10662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.447454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45030)
0809 16:29:59.447544 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:59.447626 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 16:29:59.447841 9364 leader_election.cc:334] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975): Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
I20260809 16:29:59.448104 10666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.447874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45038)
0809 16:29:59.448027 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:59.448067 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260809 16:29:59.448222 9713 leader_election.cc:334] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975): Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
I20260809 16:29:59.450912 10663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.446622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:35459 (local address 127.0.0.1:59416)
0809 16:29:59.446866 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:59.446877 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:59.446942 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:59.447158 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:59.447159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:59.447162 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:59.447213 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:59.447215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:59.448361 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:59.448362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:59.449770 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:59.449774 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:59.450600 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:59.450602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:59.450675 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:59.450680 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:29:59.450747 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:29:59.450796 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":41,"threads_started":1}
I20260809 16:29:59.450912 10661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.446506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:46157 (local address 127.0.0.1:56084)
0809 16:29:59.446840 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:59.446852 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:29:59.446923 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:29:59.447084 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:29:59.447085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:29:59.447088 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:29:59.447149 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:59.447158 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:59.448494 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:59.448496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:59.449828 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:29:59.449832 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:29:59.450600 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:29:59.450602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:29:59.450675 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:59.450680 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:29:59.450747 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:29:59.450802 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":60,"threads_started":1}
I20260809 16:29:59.450914 10667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.446719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56084 (local address 127.30.23.188:46157)
0809 16:29:59.446909 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:59.446912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:59.446923 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:29:59.446991 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:59.446992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:59.447004 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:59.447072 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:59.447414 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:59.448418 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:59.449909 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:59.450486 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:59.450569 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:59.450581 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:59.450775 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 16:29:59.450821 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":40,"threads_started":1}
I20260809 16:29:59.450915 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.446719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59416 (local address 127.30.23.189:35459)
0809 16:29:59.446814 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 16:29:59.446817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:29:59.446998 (+ 181us) server_negotiation.cc:407] Connection header received
0809 16:29:59.447026 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:29:59.447027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:29:59.447037 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:29:59.447107 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:29:59.447288 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:59.448219 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:59.449899 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:29:59.450486 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:29:59.450569 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:29:59.450581 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 16:29:59.450775 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 16:29:59.450820 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":41,"threads_started":1}
I20260809 16:29:59.451570 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb08293159084c60b56ecd9abe325431" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8d9c32a955cf44c091f52611fe665054" is_pre_election: true
I20260809 16:29:59.451578 9762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d9c32a955cf44c091f52611fe665054" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bb08293159084c60b56ecd9abe325431" is_pre_election: true
I20260809 16:29:59.451730 9762 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d9c32a955cf44c091f52611fe665054 in term 1.
I20260809 16:29:59.451730 9416 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb08293159084c60b56ecd9abe325431 in term 1.
I20260809 16:29:59.451927 9368 leader_election.cc:302] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d9c32a955cf44c091f52611fe665054, bb08293159084c60b56ecd9abe325431; no voters: 1d1d69d1116e40e29f5dfb7ec05c37b3
I20260809 16:29:59.451931 9715 leader_election.cc:302] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d9c32a955cf44c091f52611fe665054, bb08293159084c60b56ecd9abe325431; no voters: 1d1d69d1116e40e29f5dfb7ec05c37b3
I20260809 16:29:59.452044 10659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 16:29:59.452040 10660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 16:29:59.452095 10660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1d1d69d1116e40e29f5dfb7ec05c37b3)
I20260809 16:29:59.452095 10659 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1d1d69d1116e40e29f5dfb7ec05c37b3)
I20260809 16:29:59.452126 10659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:29:59.452126 10660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:29:59.453061 10659 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } }
I20260809 16:29:59.453061 10660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } }
I20260809 16:29:59.453220 10659 leader_election.cc:288] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [CANDIDATE]: Term 2 election: Requested vote from peers 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975), 8d9c32a955cf44c091f52611fe665054 (127.30.23.189:35459)
I20260809 16:29:59.453230 10660 leader_election.cc:288] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 2 election: Requested vote from peers 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975), bb08293159084c60b56ecd9abe325431 (127.30.23.188:46157)
I20260809 16:29:59.453404 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb08293159084c60b56ecd9abe325431" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8d9c32a955cf44c091f52611fe665054"
I20260809 16:29:59.453519 9416 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bb08293159084c60b56ecd9abe325431 in current term 2: Already voted for candidate 8d9c32a955cf44c091f52611fe665054 in this term.
I20260809 16:29:59.453594 10663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.453462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45052)
0809 16:29:59.453524 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:59.453563 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 16:29:59.453642 9762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d9c32a955cf44c091f52611fe665054" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bb08293159084c60b56ecd9abe325431"
I20260809 16:29:59.453725 9762 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d9c32a955cf44c091f52611fe665054 in current term 2: Already voted for candidate bb08293159084c60b56ecd9abe325431 in this term.
I20260809 16:29:59.454041 10663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.453970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45068)
0809 16:29:59.454000 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:59.454024 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 16:29:59.454114 9713 leader_election.cc:334] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975): Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
I20260809 16:29:59.454172 9713 leader_election.cc:302] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb08293159084c60b56ecd9abe325431; no voters: 1d1d69d1116e40e29f5dfb7ec05c37b3, 8d9c32a955cf44c091f52611fe665054
I20260809 16:29:59.454255 10659 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 16:29:59.454066 10661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.453317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45042)
0809 16:29:59.454013 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:59.454042 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":679}
I20260809 16:29:59.454589 10661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.454515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45072)
0809 16:29:59.454541 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:59.454565 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260809 16:29:59.454663 9364 leader_election.cc:334] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975): Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
I20260809 16:29:59.454748 9364 leader_election.cc:302] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d9c32a955cf44c091f52611fe665054; no voters: 1d1d69d1116e40e29f5dfb7ec05c37b3, bb08293159084c60b56ecd9abe325431
I20260809 16:29:59.454838 10660 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 16:29:59.942709 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.942412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.129:50361)
0809 16:29:59.942595 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:59.942646 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":112,"threads_started":1}
I20260809 16:29:59.943516 10677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.943295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.131:52953)
0809 16:29:59.943413 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:59.943463 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260809 16:29:59.944754 10681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:29:59.944543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.130:50069)
0809 16:29:59.944666 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 16:29:59.944696 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":82,"threads_started":1}
I20260809 16:30:00.943639 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:00.943286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.129:49447)
0809 16:30:00.943452 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:00.943525 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260809 16:30:00.944191 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:00.943873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.131:45997)
0809 16:30:00.944097 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:00.944142 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":110,"threads_started":1}
I20260809 16:30:00.945587 10684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:00.945388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.130:58911)
0809 16:30:00.945498 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:00.945540 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":61,"threads_started":1}
I20260809 16:30:01.539151 10685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:30:01.539261 10685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } }
I20260809 16:30:01.539464 10685 leader_election.cc:288] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975), bb08293159084c60b56ecd9abe325431 (127.30.23.188:46157)
I20260809 16:30:01.539746 9762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d9c32a955cf44c091f52611fe665054" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bb08293159084c60b56ecd9abe325431" is_pre_election: true
I20260809 16:30:01.539829 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.539612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45376)
0809 16:30:01.539744 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.539788 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260809 16:30:01.539857 9762 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d9c32a955cf44c091f52611fe665054 in term 2.
I20260809 16:30:01.540028 9368 leader_election.cc:302] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d9c32a955cf44c091f52611fe665054, bb08293159084c60b56ecd9abe325431; no voters:
I20260809 16:30:01.540118 10685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 16:30:01.540162 10685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:30:01.540189 10685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 2 FOLLOWER]: Advancing to term 3
I20260809 16:30:01.540366 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.540275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45390)
0809 16:30:01.540312 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.540343 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260809 16:30:01.540455 9364 leader_election.cc:334] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975): Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
I20260809 16:30:01.541015 10685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } }
I20260809 16:30:01.541162 10685 leader_election.cc:288] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 3 election: Requested vote from peers 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975), bb08293159084c60b56ecd9abe325431 (127.30.23.188:46157)
I20260809 16:30:01.541319 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.541237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45406)
0809 16:30:01.541272 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.541304 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 16:30:01.541318 9762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d9c32a955cf44c091f52611fe665054" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bb08293159084c60b56ecd9abe325431"
I20260809 16:30:01.541482 9762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 2 FOLLOWER]: Advancing to term 3
I20260809 16:30:01.541635 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.541560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45418)
0809 16:30:01.541590 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.541620 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260809 16:30:01.541707 9364 leader_election.cc:334] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975): Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
I20260809 16:30:01.542230 9762 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d9c32a955cf44c091f52611fe665054 in term 3.
I20260809 16:30:01.542408 9368 leader_election.cc:302] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8d9c32a955cf44c091f52611fe665054, bb08293159084c60b56ecd9abe325431; no voters: 1d1d69d1116e40e29f5dfb7ec05c37b3
I20260809 16:30:01.542526 10685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 16:30:01.542646 10685 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [term 3 LEADER]: Becoming Leader. State: Replica: 8d9c32a955cf44c091f52611fe665054, State: Running, Role: LEADER
I20260809 16:30:01.542729 10685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } }
I20260809 16:30:01.543146 10689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d9c32a955cf44c091f52611fe665054. Latest consensus state: current_term: 3 leader_uuid: "8d9c32a955cf44c091f52611fe665054" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } } }
I20260809 16:30:01.543251 10689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: This master's current role is: LEADER
I20260809 16:30:01.543443 10691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:30:01.543833 10691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:30:01.543885 10691 catalog_manager.cc:1259] Loaded cluster ID: b0b8579f5eec4a569d041b1164b4e8b0
I20260809 16:30:01.543902 10691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:30:01.544108 10691 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:30:01.544194 10691 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: Loaded TSK: 0
I20260809 16:30:01.544301 10691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:30:01.550922 9396 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:30:01.552642 9396 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:30:01.553812 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.553695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45430)
0809 16:30:01.553748 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.553784 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 16:30:01.553838 9762 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [term 3 FOLLOWER]: Refusing update from remote peer 8d9c32a955cf44c091f52611fe665054: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 16:30:01.554082 10685 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 16:30:01.554255 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.554152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.0.0.1:45434)
0809 16:30:01.554200 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.554228 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 16:30:01.554370 9364 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 -> Peer 1d1d69d1116e40e29f5dfb7ec05c37b3 (127.30.23.190:35975): Couldn't send request to peer 1d1d69d1116e40e29f5dfb7ec05c37b3. Status: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 16:30:01.554725 10695 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d9c32a955cf44c091f52611fe665054. Latest consensus state: current_term: 3 leader_uuid: "8d9c32a955cf44c091f52611fe665054" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } } }
I20260809 16:30:01.554843 10695 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:30:01.555296 10689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "8d9c32a955cf44c091f52611fe665054" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } } }
I20260809 16:30:01.555382 10689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054 [sys.catalog]: This master's current role is: LEADER
I20260809 16:30:01.555372 10695 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "8d9c32a955cf44c091f52611fe665054" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1d1d69d1116e40e29f5dfb7ec05c37b3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35975 } } peers { permanent_uuid: "8d9c32a955cf44c091f52611fe665054" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 35459 } attrs { promote: false } } peers { permanent_uuid: "bb08293159084c60b56ecd9abe325431" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 46157 } attrs { promote: false } } }
I20260809 16:30:01.555459 10695 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb08293159084c60b56ecd9abe325431 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:30:01.555575 10698 mvcc.cc:205] Tried to move back new op lower bound from 7316656134361956352 to 7316656134319689728. Current Snapshot: MvccSnapshot[applied={T|T < 7316656134361956352}]
I20260809 16:30:01.560894 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.557643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57170 (local address 127.30.23.131:35311)
0809 16:30:01.557790 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:01.557793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:01.557802 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:30:01.557833 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:01.557834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:01.557862 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:01.557929 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:01.558145 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.558788 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.560050 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.560566 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.560627 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:01.560640 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:01.560771 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 16:30:01.560820 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":49,"threads_started":1}
I20260809 16:30:01.560892 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.557453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:35311 (local address 127.0.0.1:57170)
0809 16:30:01.557600 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.557681 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:01.557738 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:01.557988 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:01.557989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:01.557991 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:01.558062 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:01.558064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.558896 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.558896 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:01.559952 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:01.559954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.560635 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.560636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:01.560703 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:01.560707 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:30:01.560758 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:30:01.560802 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260809 16:30:01.561399 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.557341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:45517 (local address 127.0.0.1:37768)
0809 16:30:01.557523 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.557535 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:01.557589 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:01.558143 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:01.558143 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:01.558145 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:01.558189 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:01.558191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.559220 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.559221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:01.560250 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:01.560252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.561122 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.561123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:01.561190 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:01.561194 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:30:01.561245 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:30:01.561285 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260809 16:30:01.561440 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.557425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37768 (local address 127.30.23.130:45517)
0809 16:30:01.558015 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:01.558017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:01.558026 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:30:01.558053 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:01.558054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:01.558066 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:01.558137 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:01.558272 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.559134 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.560342 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.561006 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.561076 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:01.561088 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:01.561271 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 16:30:01.561332 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":62,"threads_started":1}
I20260809 16:30:01.561465 10401 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6c37720fba4e6ea5087b55b66c1122 (DEFAULT_TABLE table=test-table [id=b2cdd937305e442d86c83edfbba473a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:01.561622 10401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6c37720fba4e6ea5087b55b66c1122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:01.561547 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.557453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:33891 (local address 127.0.0.1:39370)
0809 16:30:01.557829 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.557838 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:01.557882 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:01.558380 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:01.558381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:01.558383 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:01.558431 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:01.558432 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.559412 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.559413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:01.560478 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:01.560481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.561306 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.561307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:01.561393 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:01.561398 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:30:01.561429 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 16:30:01.561475 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":3,"thread_start_us":86,"threads_started":1}
I20260809 16:30:01.561961 10185 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6c37720fba4e6ea5087b55b66c1122 (DEFAULT_TABLE table=test-table [id=b2cdd937305e442d86c83edfbba473a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:01.562078 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.557500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39370 (local address 127.30.23.129:33891)
0809 16:30:01.558223 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:01.558241 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:01.558251 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:30:01.558278 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:01.558279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:01.558289 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:01.558376 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:01.558566 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.559326 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.560576 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.561206 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.561653 (+ 447us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:01.561686 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:01.561706 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 16:30:01.561995 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":65,"threads_started":1}
I20260809 16:30:01.562093 10185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6c37720fba4e6ea5087b55b66c1122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:01.562268 9950 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6c37720fba4e6ea5087b55b66c1122 (DEFAULT_TABLE table=test-table [id=b2cdd937305e442d86c83edfbba473a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:01.562465 9950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6c37720fba4e6ea5087b55b66c1122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:01.563361 10711 tablet_bootstrap.cc:492] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb: Bootstrap starting.
I20260809 16:30:01.563787 10711 tablet_bootstrap.cc:654] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:01.563869 10712 tablet_bootstrap.cc:492] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2: Bootstrap starting.
I20260809 16:30:01.564142 10714 tablet_bootstrap.cc:492] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738: Bootstrap starting.
I20260809 16:30:01.564322 10712 tablet_bootstrap.cc:654] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:01.564563 10714 tablet_bootstrap.cc:654] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:01.564941 10711 tablet_bootstrap.cc:492] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb: No bootstrap required, opened a new log
I20260809 16:30:01.564976 10712 tablet_bootstrap.cc:492] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2: No bootstrap required, opened a new log
I20260809 16:30:01.565011 10711 ts_tablet_manager.cc:1397] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:30:01.565013 10712 ts_tablet_manager.cc:1397] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:01.565141 10711 raft_consensus.cc:348] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } }
I20260809 16:30:01.565166 10712 raft_consensus.cc:348] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } }
I20260809 16:30:01.565215 10712 raft_consensus.cc:374] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:01.565215 10711 raft_consensus.cc:374] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:01.565228 10712 raft_consensus.cc:729] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e682250219344f09ef1b9bd92b8a2f2, State: Initialized, Role: FOLLOWER
I20260809 16:30:01.565240 10711 raft_consensus.cc:729] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a81ae62aedb4143a7ecd70c9e9742fb, State: Initialized, Role: FOLLOWER
I20260809 16:30:01.565300 10711 consensus_queue.cc:260] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } }
I20260809 16:30:01.565294 10712 consensus_queue.cc:260] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } }
I20260809 16:30:01.565483 10711 ts_tablet_manager.cc:1428] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:30:01.565490 10712 ts_tablet_manager.cc:1428] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:30:01.565846 10714 tablet_bootstrap.cc:492] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738: No bootstrap required, opened a new log
I20260809 16:30:01.565915 10714 ts_tablet_manager.cc:1397] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:30:01.566053 10714 raft_consensus.cc:348] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } }
I20260809 16:30:01.566123 10714 raft_consensus.cc:374] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:01.566146 10714 raft_consensus.cc:729] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 110100e55f1e404bba5a298ef2699738, State: Initialized, Role: FOLLOWER
I20260809 16:30:01.566136 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.565871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.129:50513)
0809 16:30:01.566047 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.566086 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":113,"threads_started":1}
I20260809 16:30:01.566192 10714 consensus_queue.cc:260] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } }
I20260809 16:30:01.566357 10714 ts_tablet_manager.cc:1428] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:01.566666 10066 heartbeater.cc:503] Master 127.30.23.189:35459 was elected leader, sending a full tablet report...
I20260809 16:30:01.566694 10534 heartbeater.cc:503] Master 127.30.23.189:35459 was elected leader, sending a full tablet report...
I20260809 16:30:01.567058 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.566782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.130:59441)
0809 16:30:01.566975 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.567012 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":76,"threads_started":1}
I20260809 16:30:01.567193 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.566926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.129:40549)
0809 16:30:01.567051 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.567114 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 16:30:01.567657 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.566110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.131:42391)
0809 16:30:01.567588 (+ 1478us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.567627 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1457,"thread_start_us":80,"threads_started":1}
I20260809 16:30:01.567819 10283 heartbeater.cc:503] Master 127.30.23.189:35459 was elected leader, sending a full tablet report...
I20260809 16:30:01.568001 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.567910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.131:56419)
0809 16:30:01.567949 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.567972 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 16:30:01.568011 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.567931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.130:46641)
0809 16:30:01.567966 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.567991 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 16:30:01.679209 10720 raft_consensus.cc:482] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:30:01.679342 10720 raft_consensus.cc:504] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } }
I20260809 16:30:01.679752 10720 leader_election.cc:288] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 110100e55f1e404bba5a298ef2699738 (127.30.23.130:45517), 0a81ae62aedb4143a7ecd70c9e9742fb (127.30.23.131:35311)
I20260809 16:30:01.683566 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.680002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:58263 (local address 127.30.23.130:45517)
0809 16:30:01.680157 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:01.680159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:01.680168 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:30:01.680203 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:01.680204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:01.680212 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:01.680279 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:01.680503 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.681445 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.682513 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.683118 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.683200 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:01.683211 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:01.683445 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 16:30:01.683492 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260809 16:30:01.683614 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.679902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:45517 (local address 127.30.23.129:58263)
0809 16:30:01.680043 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.680058 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:01.680118 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:01.680285 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:01.680285 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:01.680289 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:01.680408 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:01.680410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.681568 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.681568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:01.682404 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:01.682407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.683229 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.683230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:01.683348 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:01.683354 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:30:01.683430 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 16:30:01.683486 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 16:30:01.683916 10211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6c37720fba4e6ea5087b55b66c1122" candidate_uuid: "3e682250219344f09ef1b9bd92b8a2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110100e55f1e404bba5a298ef2699738" is_pre_election: true
I20260809 16:30:01.684055 10211 raft_consensus.cc:2454] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e682250219344f09ef1b9bd92b8a2f2 in term 0.
I20260809 16:30:01.684279 9859 leader_election.cc:302] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 110100e55f1e404bba5a298ef2699738, 3e682250219344f09ef1b9bd92b8a2f2; no voters:
I20260809 16:30:01.684458 10720 raft_consensus.cc:2781] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:30:01.684504 10720 raft_consensus.cc:482] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:30:01.684526 10720 raft_consensus.cc:3037] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:01.684768 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.679960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:35311 (local address 127.30.23.129:40231)
0809 16:30:01.680192 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.680203 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:01.680319 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:01.681463 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:01.681464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:01.681467 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:01.681562 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:01.681564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.682638 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.682639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:01.683681 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:01.683685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.684500 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.684501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:01.684581 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:01.684585 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:30:01.684638 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 16:30:01.684686 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":132,"threads_started":1}
I20260809 16:30:01.684834 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.680069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:40231 (local address 127.30.23.131:35311)
0809 16:30:01.680121 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:01.680123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:01.680270 (+ 147us) server_negotiation.cc:407] Connection header received
0809 16:30:01.680405 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:01.680406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:01.680418 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:01.680476 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:01.681687 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.682533 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.683796 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.684359 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.684413 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:01.684437 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:01.684667 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 16:30:01.684735 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:30:01.685153 10437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6c37720fba4e6ea5087b55b66c1122" candidate_uuid: "3e682250219344f09ef1b9bd92b8a2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" is_pre_election: true
I20260809 16:30:01.685304 10437 raft_consensus.cc:2454] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e682250219344f09ef1b9bd92b8a2f2 in term 0.
I20260809 16:30:01.685814 10720 raft_consensus.cc:504] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } }
I20260809 16:30:01.686002 10720 leader_election.cc:288] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [CANDIDATE]: Term 1 election: Requested vote from peers 110100e55f1e404bba5a298ef2699738 (127.30.23.130:45517), 0a81ae62aedb4143a7ecd70c9e9742fb (127.30.23.131:35311)
I20260809 16:30:01.686187 10211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6c37720fba4e6ea5087b55b66c1122" candidate_uuid: "3e682250219344f09ef1b9bd92b8a2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110100e55f1e404bba5a298ef2699738"
I20260809 16:30:01.686293 10211 raft_consensus.cc:3037] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:01.687332 10211 raft_consensus.cc:2454] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e682250219344f09ef1b9bd92b8a2f2 in term 1.
I20260809 16:30:01.687469 10437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6c37720fba4e6ea5087b55b66c1122" candidate_uuid: "3e682250219344f09ef1b9bd92b8a2f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb"
I20260809 16:30:01.687559 10437 raft_consensus.cc:3037] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:01.687654 9859 leader_election.cc:302] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 110100e55f1e404bba5a298ef2699738, 3e682250219344f09ef1b9bd92b8a2f2; no voters:
I20260809 16:30:01.687752 10720 raft_consensus.cc:2781] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:30:01.687930 10720 raft_consensus.cc:686] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [term 1 LEADER]: Becoming Leader. State: Replica: 3e682250219344f09ef1b9bd92b8a2f2, State: Running, Role: LEADER
I20260809 16:30:01.688005 10720 consensus_queue.cc:237] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } } peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } }
I20260809 16:30:01.688592 10437 raft_consensus.cc:2454] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e682250219344f09ef1b9bd92b8a2f2 in term 1.
I20260809 16:30:01.688711 9396 catalog_manager.cc:5495] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e682250219344f09ef1b9bd92b8a2f2 (127.30.23.129). New cstate: current_term: 1 leader_uuid: "3e682250219344f09ef1b9bd92b8a2f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e682250219344f09ef1b9bd92b8a2f2" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 } health_report { overall_health: UNKNOWN } } }
I20260809 16:30:01.688982 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.688240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.129:53829)
0809 16:30:01.688915 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.688952 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200}
I20260809 16:30:01.708716 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.706270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39376 (local address 127.30.23.129:33891)
0809 16:30:01.706332 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:01.706334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:01.706343 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:30:01.706392 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:01.706392 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:01.706412 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:01.706477 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:30:01.706624 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.707173 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.707958 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:01.708152 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:01.708224 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:01.708377 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:30:01.708505 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:30:01.708570 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:01.708637 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 16:30:01.708659 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:30:01.709854 10211 raft_consensus.cc:1261] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738 [term 1 FOLLOWER]: Refusing update from remote peer 3e682250219344f09ef1b9bd92b8a2f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:01.709928 10437 raft_consensus.cc:1261] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 1 FOLLOWER]: Refusing update from remote peer 3e682250219344f09ef1b9bd92b8a2f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:01.710139 10732 consensus_queue.cc:1035] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "110100e55f1e404bba5a298ef2699738" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 45517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:01.710266 10720 consensus_queue.cc:1035] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a81ae62aedb4143a7ecd70c9e9742fb" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:01.711336 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.711143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.131:56101)
0809 16:30:01.711254 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.711301 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 16:30:01.711815 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.711678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.130:52301)
0809 16:30:01.711756 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.711783 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 16:30:01.712514 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.712409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.129:42439)
0809 16:30:01.712450 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.712476 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 16:30:01.712873 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.712791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.129:37627)
0809 16:30:01.712820 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.712847 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260809 16:30:01.713136 10736 mvcc.cc:205] Tried to move back new op lower bound from 7316656135001092096 to 7316656134914523136. Current Snapshot: MvccSnapshot[applied={T|T < 7316656135001092096}]
I20260809 16:30:01.713635 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.713517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.131:44659)
0809 16:30:01.713571 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.713600 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 16:30:01.713974 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.713885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:35975 (local address 127.30.23.130:36359)
0809 16:30:01.713925 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.713948 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:35975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 16:30:01.714777 9395 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:30:01.714874 9395 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:30:01.716288 9395 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d9c32a955cf44c091f52611fe665054: Sending DeleteTablet for 3 replicas of tablet 1f6c37720fba4e6ea5087b55b66c1122
I20260809 16:30:01.716578 10185 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f6c37720fba4e6ea5087b55b66c1122 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:01 UTC) from {username='oryx', principal='oryx/127.30.23.189@KRBTEST.COM'} at 127.0.0.1:37768
I20260809 16:30:01.716602 10401 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f6c37720fba4e6ea5087b55b66c1122 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:01 UTC) from {username='oryx', principal='oryx/127.30.23.189@KRBTEST.COM'} at 127.0.0.1:57170
I20260809 16:30:01.716689 9950 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f6c37720fba4e6ea5087b55b66c1122 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:01 UTC) from {username='oryx', principal='oryx/127.30.23.189@KRBTEST.COM'} at 127.0.0.1:39370
I20260809 16:30:01.716985 10755 tablet_replica.cc:330] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2: stopping tablet replica
I20260809 16:30:01.717047 10755 raft_consensus.cc:2229] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:30:01.717162 10755 raft_consensus.cc:2258] T 1f6c37720fba4e6ea5087b55b66c1122 P 3e682250219344f09ef1b9bd92b8a2f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:01.717175 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 9678
I20260809 16:30:01.720485 10750 tablet_replica.cc:330] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738: stopping tablet replica
I20260809 16:30:01.720631 10750 raft_consensus.cc:2229] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:30:01.720745 10750 raft_consensus.cc:2258] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:01.741464 10750 ts_tablet_manager.cc:1905] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:01.743804 10750 ts_tablet_manager.cc:1918] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:30:01.743921 10750 log.cc:1196] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/1f6c37720fba4e6ea5087b55b66c1122
I20260809 16:30:01.744225 10750 ts_tablet_manager.cc:1939] T 1f6c37720fba4e6ea5087b55b66c1122 P 110100e55f1e404bba5a298ef2699738: Deleting consensus metadata
I20260809 16:30:01.745774 9364 catalog_manager.cc:4841] TS 110100e55f1e404bba5a298ef2699738 (127.30.23.130:45517): tablet 1f6c37720fba4e6ea5087b55b66c1122 (table test-table [id=b2cdd937305e442d86c83edfbba473a9]) successfully deleted
I20260809 16:30:01.749533 10751 tablet_replica.cc:330] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb: stopping tablet replica
I20260809 16:30:01.749642 10751 raft_consensus.cc:2229] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:30:01.749735 10751 raft_consensus.cc:2258] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:01.750141 10751 ts_tablet_manager.cc:1905] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:01.751897 10751 ts_tablet_manager.cc:1918] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:30:01.751992 10751 log.cc:1196] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/1f6c37720fba4e6ea5087b55b66c1122
I20260809 16:30:01.752084 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 10078
I20260809 16:30:01.752130 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:01.752013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:33891 (local address 127.0.0.1:39386)
0809 16:30:01.752056 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:01.752086 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:33891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260809 16:30:01.768286 9372 catalog_manager.cc:4584] TS 3e682250219344f09ef1b9bd92b8a2f2 (127.30.23.129:33891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1f6c37720fba4e6ea5087b55b66c1122: Network error: Client connection negotiation failed: client connection to 127.30.23.129:33891: connect: Connection refused (error 111)
I20260809 16:30:01.777644 10751 ts_tablet_manager.cc:1939] T 1f6c37720fba4e6ea5087b55b66c1122 P 0a81ae62aedb4143a7ecd70c9e9742fb: Deleting consensus metadata
I20260809 16:30:01.778203 9370 catalog_manager.cc:4841] TS 0a81ae62aedb4143a7ecd70c9e9742fb (127.30.23.131:35311): tablet 1f6c37720fba4e6ea5087b55b66c1122 (table test-table [id=b2cdd937305e442d86c83edfbba473a9]) successfully deleted
I20260809 16:30:01.782019 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 10290
I20260809 16:30:01.802093 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 9163
I20260809 16:30:01.814030 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 9488
2026-08-09T16:30:01Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7637 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:30:01 dist-test-slave-j2vl krb5kdc[10760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:30:01 dist-test-slave-j2vl krb5kdc[10760](info): set up 2 sockets
Aug 09 16:30:01 dist-test-slave-j2vl krb5kdc[10760](info): commencing operation
krb5kdc: starting...
W20260809 16:30:03.120750 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:30:03 dist-test-slave-j2vl krb5kdc[10760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:30:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:30:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.190.keytab.
Entry for principal oryx/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.190.keytab.
I20260809 16:30:03.256599 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:44729
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.190.keytab
--principal=oryx/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:44729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:03.343853 10781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:03.344170 10781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:03.344218 10781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:03.345775 10781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:30:03.345861 10781 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 16:30:03.345886 10781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:03.345913 10781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:03.345963 10781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:30:03.346000 10781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:30:03.347872 10781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:44729
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.190.keytab
--principal=oryx/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:44729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:03.348194 10781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:03.348455 10781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:03.353673 10789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:03.353801 10788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:03.353991 10795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:03.354468 10781 server_base.cc:1034] running on GCE node
I20260809 16:30:03.354806 10781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:03.355134 10781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:03.356352 10781 hybrid_clock.cc:648] HybridClock initialized: now 1786293003356318 us; error 57 us; skew 500 ppm
Aug 09 16:30:03 dist-test-slave-j2vl krb5kdc[10760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293003, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:03.359266 10781 init.cc:377] Logged in from keytab as oryx/127.30.23.190@KRBTEST.COM (short username oryx)
I20260809 16:30:03.359697 10781 webserver.cc:460] Webserver started at http://127.30.23.190:36583/ using document root <none> and password file <none>
I20260809 16:30:03.359902 10781 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:03.359956 10781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:03.360056 10781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:03.361183 10781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "acfd262f61ed440db22479a7342d7ac7"
format_stamp: "Formatted at 2026-08-09 16:30:03 on dist-test-slave-j2vl"
server_key: "6ac9acaf8446d0da90e4fe4c502ee671"
server_key_iv: "15614d46c30f7505060fc1a452b3b086"
server_key_version: "encryptionkey@0"
I20260809 16:30:03.361593 10781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "acfd262f61ed440db22479a7342d7ac7"
format_stamp: "Formatted at 2026-08-09 16:30:03 on dist-test-slave-j2vl"
server_key: "6ac9acaf8446d0da90e4fe4c502ee671"
server_key_iv: "15614d46c30f7505060fc1a452b3b086"
server_key_version: "encryptionkey@0"
I20260809 16:30:03.363127 10781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:30:03.363919 10806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:03.364114 10781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:30:03.364202 10781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "acfd262f61ed440db22479a7342d7ac7"
format_stamp: "Formatted at 2026-08-09 16:30:03 on dist-test-slave-j2vl"
server_key: "6ac9acaf8446d0da90e4fe4c502ee671"
server_key_iv: "15614d46c30f7505060fc1a452b3b086"
server_key_version: "encryptionkey@0"
I20260809 16:30:03.364271 10781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:03.386065 10781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:03.388507 10781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:03.388629 10781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:03.393319 10781 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:44729
I20260809 16:30:03.393441 10912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:44729 every 8 connection(s)
I20260809 16:30:03.393769 10781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:30:03.394487 10913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:03.396977 10913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7: Bootstrap starting.
I20260809 16:30:03.397673 10913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:03.397943 10913 log.cc:824] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:30:03.398648 10913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7: No bootstrap required, opened a new log
I20260809 16:30:03.400897 10913 raft_consensus.cc:348] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acfd262f61ed440db22479a7342d7ac7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44729 } }
I20260809 16:30:03.401047 10913 raft_consensus.cc:374] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:03.401072 10913 raft_consensus.cc:729] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfd262f61ed440db22479a7342d7ac7, State: Initialized, Role: FOLLOWER
I20260809 16:30:03.401176 10913 consensus_queue.cc:260] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acfd262f61ed440db22479a7342d7ac7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44729 } }
I20260809 16:30:03.401271 10913 raft_consensus.cc:388] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:30:03.401314 10913 raft_consensus.cc:482] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:30:03.401410 10913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:03.402002 10913 raft_consensus.cc:504] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acfd262f61ed440db22479a7342d7ac7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44729 } }
I20260809 16:30:03.402127 10913 leader_election.cc:302] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acfd262f61ed440db22479a7342d7ac7; no voters:
I20260809 16:30:03.402357 10913 leader_election.cc:288] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:30:03.402395 10918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:30:03.402493 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 10781
I20260809 16:30:03.402526 10918 raft_consensus.cc:686] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [term 1 LEADER]: Becoming Leader. State: Replica: acfd262f61ed440db22479a7342d7ac7, State: Running, Role: LEADER
I20260809 16:30:03.402602 10918 consensus_queue.cc:237] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acfd262f61ed440db22479a7342d7ac7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44729 } }
I20260809 16:30:03.402614 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:30:03.402635 10913 sys_catalog.cc:564] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:30:03.402850 30814 external_mini_cluster.cc:1428] Setting key 40e38685ae6cfaf0baced4667a04cc5b
I20260809 16:30:03.402925 10918 sys_catalog.cc:455] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acfd262f61ed440db22479a7342d7ac7. Latest consensus state: current_term: 1 leader_uuid: "acfd262f61ed440db22479a7342d7ac7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acfd262f61ed440db22479a7342d7ac7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44729 } } }
I20260809 16:30:03.403004 10918 sys_catalog.cc:458] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [sys.catalog]: This master's current role is: LEADER
I20260809 16:30:03.402941 10920 sys_catalog.cc:455] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acfd262f61ed440db22479a7342d7ac7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acfd262f61ed440db22479a7342d7ac7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44729 } } }
I20260809 16:30:03.403050 10920 sys_catalog.cc:458] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7 [sys.catalog]: This master's current role is: LEADER
I20260809 16:30:03.404176 10930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:30:03.404738 10930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:30:03.406383 10930 catalog_manager.cc:1347] Generated new cluster ID: c453509194c2490385de59eaa35577ea
I20260809 16:30:03.406441 10930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:30:03 dist-test-slave-j2vl krb5kdc[10760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:30:03.413792 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.404156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.30.23.190:44729)
0809 16:30:03.404394 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:03.404406 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:03.404446 (+ 40us) server_negotiation.cc:407] Connection header received
0809 16:30:03.404583 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:03.404591 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:03.404668 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:03.404817 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:03.405786 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.406930 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.407661 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.408099 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.408213 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.410594 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:03.410597 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:03.410598 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:03.410617 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:03.412261 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.412639 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.412642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.412643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.412733 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.412941 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.412942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.412942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.413183 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:03.413412 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:03.413480 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 16:30:03.413623 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":87,"threads_started":1}
I20260809 16:30:03.416230 10930 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:30:03.416745 10930 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:30:03.423969 10930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7: Generated new TSK 0
I20260809 16:30:03.424182 10930 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab.
Entry for principal oryx/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab.
I20260809 16:30:03.460316 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44729
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab
--principal=oryx/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:03.550431 10953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:03.550633 10953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:03.550660 10953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:03.551913 10953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:03.551957 10953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:03.551980 10953 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:30:03.551997 10953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:30:03.553697 10953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab
--principal=oryx/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.23.190:44729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:03.553951 10953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:03.554214 10953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:03.559686 10962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:03.559708 10965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:03.559760 10963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:03.560011 10953 server_base.cc:1034] running on GCE node
I20260809 16:30:03.560248 10953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:03.560513 10953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:03.561682 10953 hybrid_clock.cc:648] HybridClock initialized: now 1786293003561669 us; error 34 us; skew 500 ppm
Aug 09 16:30:03 dist-test-slave-j2vl krb5kdc[10760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293003, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:03.564630 10953 init.cc:377] Logged in from keytab as oryx/127.30.23.129@KRBTEST.COM (short username oryx)
I20260809 16:30:03.565169 10953 webserver.cc:460] Webserver started at http://127.30.23.129:41085/ using document root <none> and password file <none>
I20260809 16:30:03.565511 10953 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:03.565587 10953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:03.565771 10953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:03.567071 10953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "eba6ed2ba52344f3b20cba56e29f8f4f"
format_stamp: "Formatted at 2026-08-09 16:30:03 on dist-test-slave-j2vl"
server_key: "f8bb2d5af74287e3f1b1a91953f55834"
server_key_iv: "a48d7366eca0ff457d38f2a636c0df26"
server_key_version: "encryptionkey@0"
I20260809 16:30:03.567457 10953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "eba6ed2ba52344f3b20cba56e29f8f4f"
format_stamp: "Formatted at 2026-08-09 16:30:03 on dist-test-slave-j2vl"
server_key: "f8bb2d5af74287e3f1b1a91953f55834"
server_key_iv: "a48d7366eca0ff457d38f2a636c0df26"
server_key_version: "encryptionkey@0"
I20260809 16:30:03.569121 10953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:30:03.570257 10980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:03.570720 10953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:03.570884 10953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "eba6ed2ba52344f3b20cba56e29f8f4f"
format_stamp: "Formatted at 2026-08-09 16:30:03 on dist-test-slave-j2vl"
server_key: "f8bb2d5af74287e3f1b1a91953f55834"
server_key_iv: "a48d7366eca0ff457d38f2a636c0df26"
server_key_version: "encryptionkey@0"
I20260809 16:30:03.570976 10953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:03.586401 10953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:03.588867 10953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:03.589063 10953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:03.589761 10953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:03.589843 10953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:03.589881 10953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:03.589911 10953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:03.597368 10953 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:45575
I20260809 16:30:03.597640 11188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:45575 every 8 connection(s)
I20260809 16:30:03.597811 10953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:30:03 dist-test-slave-j2vl krb5kdc[10760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293003, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:30:03.601512 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.590243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:57131 (local address 127.30.23.190:44729)
0809 16:30:03.590332 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:03.590334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:03.590510 (+ 176us) server_negotiation.cc:407] Connection header received
0809 16:30:03.591020 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:03.591022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:03.591037 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:03.591127 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:03.592025 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.592916 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.594245 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.594596 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.594673 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.598662 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:03.598663 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:03.598664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:03.598671 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:03.600014 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.600477 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.600479 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.600479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.600550 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.600786 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.600787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.600787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.600907 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:03.600965 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:03.601274 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 16:30:03.601435 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 16:30:03.601725 11000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.590217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44729 (local address 127.30.23.129:57131)
0809 16:30:03.590370 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:03.590412 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:03.590917 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:03.591177 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:03.591180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:03.591601 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:03.591900 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.591907 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.593039 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.593040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.594146 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.594149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.594702 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.594713 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.594900 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.596153 (+ 1253us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:03.596166 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:03.598557 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:03.600105 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:03.600113 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:03.600115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:03.600369 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:03.600622 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:03.600622 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:03.600623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:03.600727 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:03.600997 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:03.601004 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:03.601145 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 16:30:03.601248 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 16:30:03.601473 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260809 16:30:03.602283 11192 heartbeater.cc:347] Connected to a master server at 127.30.23.190:44729
I20260809 16:30:03.602386 11192 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:03.602591 11192 heartbeater.cc:511] Master 127.30.23.190:44729 requested a full tablet report, sending...
I20260809 16:30:03.603180 10837 ts_manager.cc:194] Registered new tserver with Master: eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129:45575)
I20260809 16:30:03.603888 10837 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:57131
I20260809 16:30:03.607103 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 10953
I20260809 16:30:03.607239 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:30:03.607466 30814 external_mini_cluster.cc:1428] Setting key d2910770dd68adc9db9b833379df721e
I20260809 16:30:03.610952 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.604254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:54245 (local address 127.30.23.190:44729)
0809 16:30:03.604294 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:03.604296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:03.604441 (+ 145us) server_negotiation.cc:407] Connection header received
0809 16:30:03.604518 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:03.604519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:03.604529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:03.604574 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:03.604924 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.605540 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.606302 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.606521 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.606602 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.607910 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:03.607911 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:03.607912 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:03.607919 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:03.609586 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.610095 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.610097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.610097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.610171 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.610432 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.610433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.610433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.610551 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:03.610618 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:03.610787 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 16:30:03.610882 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:30:03.611047 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.604211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44729 (local address 127.30.23.129:54245)
0809 16:30:03.604352 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:03.604363 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:03.604458 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:03.604588 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:03.604589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:03.604785 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:03.604838 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.604840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.605663 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.605664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.606211 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.606212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.606600 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.606601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.606668 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.607292 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:03.607299 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:03.607820 (+ 521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:03.609733 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:03.609734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:03.609735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:03.609947 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:03.610252 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:03.610253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:03.610254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:03.610338 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:03.610627 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:03.610628 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:03.610686 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 16:30:03.610772 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:30:03.610901 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for oryx/127.30.23.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab.
Entry for principal oryx/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab.
I20260809 16:30:03.638302 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44729
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab
--principal=oryx/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:03.725632 11208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:03.725870 11208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:03.725900 11208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:03.727098 11208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:03.727141 11208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:03.727162 11208 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:30:03.727180 11208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:30:03.728718 11208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab
--principal=oryx/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.23.190:44729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:03.728951 11208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:03.729139 11208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:03.734596 11221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:03.734767 11218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:03.734789 11219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:03.734951 11208 server_base.cc:1034] running on GCE node
I20260809 16:30:03.735457 11208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:03.735792 11208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:03.736984 11208 hybrid_clock.cc:648] HybridClock initialized: now 1786293003736994 us; error 56 us; skew 500 ppm
Aug 09 16:30:03 dist-test-slave-j2vl krb5kdc[10760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293003, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:03.739845 11208 init.cc:377] Logged in from keytab as oryx/127.30.23.130@KRBTEST.COM (short username oryx)
I20260809 16:30:03.740423 11208 webserver.cc:460] Webserver started at http://127.30.23.130:36699/ using document root <none> and password file <none>
I20260809 16:30:03.740674 11208 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:03.740743 11208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:03.740926 11208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:03.742103 11208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "a9b60fe0d3864708862a8c5256a586fd"
format_stamp: "Formatted at 2026-08-09 16:30:03 on dist-test-slave-j2vl"
server_key: "1c5c281a2d17d43013e509730b09a96b"
server_key_iv: "640a1bbf322c73be4a832edcfb322bc8"
server_key_version: "encryptionkey@0"
I20260809 16:30:03.742486 11208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "a9b60fe0d3864708862a8c5256a586fd"
format_stamp: "Formatted at 2026-08-09 16:30:03 on dist-test-slave-j2vl"
server_key: "1c5c281a2d17d43013e509730b09a96b"
server_key_iv: "640a1bbf322c73be4a832edcfb322bc8"
server_key_version: "encryptionkey@0"
I20260809 16:30:03.744072 11208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:30:03.744923 11232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:03.745184 11208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:03.745332 11208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "a9b60fe0d3864708862a8c5256a586fd"
format_stamp: "Formatted at 2026-08-09 16:30:03 on dist-test-slave-j2vl"
server_key: "1c5c281a2d17d43013e509730b09a96b"
server_key_iv: "640a1bbf322c73be4a832edcfb322bc8"
server_key_version: "encryptionkey@0"
I20260809 16:30:03.745471 11208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:03.762135 11208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:03.764592 11208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:03.764771 11208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:03.765506 11208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:03.765575 11208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:03.765640 11208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:03.765688 11208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:03.772677 11208 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:34291
I20260809 16:30:03.772776 11445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:34291 every 8 connection(s)
Aug 09 16:30:03 dist-test-slave-j2vl krb5kdc[10760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293003, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:30:03.773178 11208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:30:03.774673 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 11208
I20260809 16:30:03.774832 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:30:03.775136 30814 external_mini_cluster.cc:1428] Setting key 36760230073dfe1a39cf235921238341
I20260809 16:30:03.777190 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.765925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:42117 (local address 127.30.23.190:44729)
0809 16:30:03.765987 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:03.765990 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:03.766185 (+ 195us) server_negotiation.cc:407] Connection header received
0809 16:30:03.766791 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:03.766793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:03.766807 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:03.766862 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:03.767713 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.768585 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.770259 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.770633 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.770728 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.773784 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:03.773786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:03.773786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:03.773793 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:03.775632 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.776136 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.776138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.776138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.776212 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.776486 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.776488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.776488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.776619 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:03.776697 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:03.777008 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 16:30:03.777099 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:30:03.777405 11246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.765909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44729 (local address 127.30.23.130:42117)
0809 16:30:03.766067 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:03.766108 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:03.766685 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:03.766955 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:03.766959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:03.767304 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:03.767604 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.767610 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.768673 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.768675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.770128 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.770131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.770758 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.770760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.770949 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.771615 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:03.771625 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:03.773655 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:03.775736 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:03.775745 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:03.775748 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:03.776050 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:03.776283 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:03.776284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:03.776285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:03.776407 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:03.776688 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:03.776696 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:03.776853 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 16:30:03.776970 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 16:30:03.777143 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260809 16:30:03.778075 11447 heartbeater.cc:347] Connected to a master server at 127.30.23.190:44729
I20260809 16:30:03.778174 11447 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:03.778415 11447 heartbeater.cc:511] Master 127.30.23.190:44729 requested a full tablet report, sending...
I20260809 16:30:03.779255 10837 ts_manager.cc:194] Registered new tserver with Master: a9b60fe0d3864708862a8c5256a586fd (127.30.23.130:34291)
I20260809 16:30:03.779649 10837 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:42117
WARNING: no policy specified for oryx/127.30.23.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.23.131@KRBTEST.COM" created.
I20260809 16:30:03.788007 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.779271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:36591 (local address 127.30.23.190:44729)
0809 16:30:03.779313 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:03.779315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:03.781564 (+ 2249us) server_negotiation.cc:407] Connection header received
0809 16:30:03.781645 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:03.781646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:03.781665 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:03.781740 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:03.782130 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.782615 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.783355 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.783546 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.783625 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.784864 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:03.784866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:03.784866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:03.784882 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:03.786591 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.786997 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.786998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.786999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.787061 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.787318 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.787319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.787319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.787486 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:03.787551 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:03.787813 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 16:30:03.787939 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:30:03.788005 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.779214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44729 (local address 127.30.23.130:36591)
0809 16:30:03.781478 (+ 2264us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:03.781490 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:03.781581 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:03.781749 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:03.781750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:03.781980 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:03.782035 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.782037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.782718 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.782719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.783246 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.783248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.783625 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.783626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.783700 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.784430 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:03.784435 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:03.784741 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:03.786713 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:03.786718 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:03.786718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:03.786905 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:03.787161 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:03.787161 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:03.787162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:03.787237 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:03.787578 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:03.787579 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:03.787633 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 16:30:03.787735 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 16:30:03.787839 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2236,"thread_start_us":69,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab.
Entry for principal oryx/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab.
I20260809 16:30:03.806011 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44729
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab
--principal=oryx/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:03.894404 11466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:03.894631 11466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:03.894658 11466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:03.895803 11466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:03.895843 11466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:03.895864 11466 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:30:03.895888 11466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:30:03.897388 11466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab
--principal=oryx/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.23.190:44729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:03.897588 11466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:03.897782 11466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:03.903182 11475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:03.903283 11476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:03.903348 11466 server_base.cc:1034] running on GCE node
W20260809 16:30:03.903558 11479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:03.903825 11466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:03.904155 11466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:03.905395 11466 hybrid_clock.cc:648] HybridClock initialized: now 1786293003905316 us; error 35 us; skew 500 ppm
Aug 09 16:30:03 dist-test-slave-j2vl krb5kdc[10760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293003, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:03.908295 11466 init.cc:377] Logged in from keytab as oryx/127.30.23.131@KRBTEST.COM (short username oryx)
I20260809 16:30:03.908838 11466 webserver.cc:460] Webserver started at http://127.30.23.131:40949/ using document root <none> and password file <none>
I20260809 16:30:03.909086 11466 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:03.909132 11466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:03.909296 11466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:03.910646 11466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "44ba16574fc54f1180edcab741b6d4bf"
format_stamp: "Formatted at 2026-08-09 16:30:03 on dist-test-slave-j2vl"
server_key: "ebc9c9d00d394d9e6bd227dbaac250a9"
server_key_iv: "4564210ae17aee152193ace89393f5a9"
server_key_version: "encryptionkey@0"
I20260809 16:30:03.911062 11466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "44ba16574fc54f1180edcab741b6d4bf"
format_stamp: "Formatted at 2026-08-09 16:30:03 on dist-test-slave-j2vl"
server_key: "ebc9c9d00d394d9e6bd227dbaac250a9"
server_key_iv: "4564210ae17aee152193ace89393f5a9"
server_key_version: "encryptionkey@0"
I20260809 16:30:03.912472 11466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:30:03.913307 11496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:03.913611 11466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:03.913733 11466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "44ba16574fc54f1180edcab741b6d4bf"
format_stamp: "Formatted at 2026-08-09 16:30:03 on dist-test-slave-j2vl"
server_key: "ebc9c9d00d394d9e6bd227dbaac250a9"
server_key_iv: "4564210ae17aee152193ace89393f5a9"
server_key_version: "encryptionkey@0"
I20260809 16:30:03.913815 11466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:03.934046 11466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:03.936601 11466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:03.936790 11466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:03.937433 11466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:03.937495 11466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:03.937547 11466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:03.937597 11466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:30:03 dist-test-slave-j2vl krb5kdc[10760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293003, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:30:03.945564 11466 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:36445
I20260809 16:30:03.945619 11685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:36445 every 8 connection(s)
I20260809 16:30:03.945959 11466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:30:03.948654 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.938155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:36611 (local address 127.30.23.190:44729)
0809 16:30:03.938245 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:03.938248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:03.938424 (+ 176us) server_negotiation.cc:407] Connection header received
0809 16:30:03.939069 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:03.939071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:03.939086 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:03.939161 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:03.940248 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.941116 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.942917 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.943185 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.943281 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.945775 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:03.945777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:03.945777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:03.945785 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:03.947339 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.947738 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.947739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.947740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.947812 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.948039 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.948040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.948040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.948154 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:03.948210 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:03.948514 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 16:30:03.948588 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:30:03.948815 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.938110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44729 (local address 127.30.23.131:36611)
0809 16:30:03.938298 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:03.938336 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:03.938947 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:03.939271 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:03.939277 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:03.939784 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:03.940114 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.940122 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.941267 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.941268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.942784 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.942787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.943306 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.943307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.943456 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.944090 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:03.944101 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:03.945645 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:03.947430 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:03.947436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:03.947438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:03.947659 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:03.947875 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:03.947876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:03.947876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:03.947977 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:03.948211 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:03.948216 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:03.948360 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 16:30:03.948487 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 16:30:03.948616 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260809 16:30:03.949333 11687 heartbeater.cc:347] Connected to a master server at 127.30.23.190:44729
I20260809 16:30:03.949486 11687 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:03.949681 11687 heartbeater.cc:511] Master 127.30.23.190:44729 requested a full tablet report, sending...
I20260809 16:30:03.950141 10836 ts_manager.cc:194] Registered new tserver with Master: 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131:36445)
I20260809 16:30:03.950676 10836 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:36611
I20260809 16:30:03.952991 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 11466
I20260809 16:30:03.953141 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:30:03.953758 30814 external_mini_cluster.cc:1428] Setting key c1e3e3fa271367b441f80df180e87a83
I20260809 16:30:03.954669 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:30:03.957012 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.950833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:44897 (local address 127.30.23.190:44729)
0809 16:30:03.950872 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:03.950874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:03.950882 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:30:03.950901 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:03.950902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:03.950912 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:03.950967 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:03.951324 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.951881 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.952593 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.952797 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.952863 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.954136 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:03.954138 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:03.954139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:03.954148 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:03.955644 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.956053 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.956054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.956055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.956118 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.956402 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.956403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.956403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.956547 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:03.956638 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:03.956825 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 16:30:03.956915 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:30:03.957027 11698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.950461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44729 (local address 127.30.23.131:44897)
0809 16:30:03.950591 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:03.950602 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:03.950647 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:03.950989 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:03.950990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:03.951182 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:03.951239 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.951241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.951995 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.951996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.952496 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.952498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.952894 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.952895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.952976 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.953536 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:03.953542 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:03.954027 (+ 485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:03.955762 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:03.955763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:03.955764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:03.955949 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:03.956184 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:03.956185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:03.956185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:03.956261 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:03.956630 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:03.956631 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:03.956685 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 16:30:03.956796 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 16:30:03.956918 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
Aug 09 16:30:03 dist-test-slave-j2vl krb5kdc[10760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293003, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:30:03 dist-test-slave-j2vl krb5kdc[10760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:03 dist-test-slave-j2vl krb5kdc[10760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 16:30:03.979332 10932 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:03.973183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39824 (local address 127.30.23.190:44729)
0809 16:30:03.973252 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:03.973254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:03.973263 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:30:03.973436 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:03.973437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:03.973448 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:03.973501 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:03.974131 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.974742 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.975524 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.975722 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.975787 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.979227 (+ 3440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39824: BlockingRecv error: recv got EOF from 127.0.0.1:39824 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 16:30:03 dist-test-slave-j2vl krb5kdc[10760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293003, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:30:03.986497 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.980181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39826 (local address 127.30.23.190:44729)
0809 16:30:03.980239 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:03.980241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:03.980316 (+ 75us) server_negotiation.cc:407] Connection header received
0809 16:30:03.980356 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:03.980357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:03.980365 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:03.980422 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:03.980845 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.981325 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.982035 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.982211 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.982260 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.984340 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:03.984341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:03.984341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:03.984347 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:03.985610 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.985924 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.985925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.985925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.985975 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:03.986137 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:03.986137 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:03.986137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:03.986232 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:03.986305 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:03.986383 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 16:30:03.986436 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:30:03.988288 10836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:30:03.990070 10836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:30:03.996873 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.992658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45575 (local address 127.0.0.1:47362)
0809 16:30:03.992840 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:03.992880 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:03.992953 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:03.993436 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:03.993438 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:03.993440 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:03.993504 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.993506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.994431 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.994432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.995766 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.995769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.996533 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.996534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.996618 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.996632 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:30:03.996712 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 16:30:03.996766 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":26,"thread_start_us":70,"threads_started":1}
I20260809 16:30:03.996989 11728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.992658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:36445 (local address 127.0.0.1:55160)
0809 16:30:03.992888 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:03.992900 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:03.992957 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:03.993303 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:03.993306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:03.993308 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:03.993398 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.993400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.994753 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.994754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.995826 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.995828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.996713 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.996715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.996786 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.996789 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:30:03.996844 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:30:03.996877 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":7,"spinlock_wait_cycles":768,"thread_start_us":136,"threads_started":1}
I20260809 16:30:03.997254 11730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.992873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55160 (local address 127.30.23.131:36445)
0809 16:30:03.993099 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:03.993105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:03.993117 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:30:03.993159 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:03.993162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:03.993179 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:03.993267 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:03.993498 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.994637 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.995919 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.996603 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.996691 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.996778 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:03.996867 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 16:30:03.996909 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":106,"threads_started":1}
I20260809 16:30:03.997387 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.992673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:34291 (local address 127.0.0.1:45296)
0809 16:30:03.993610 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:03.993621 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:03.993672 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:03.993929 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:03.993930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:03.993931 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:03.993969 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.993971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.995048 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.995049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.996004 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:03.996007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.996996 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.996998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:03.997056 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.997060 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:30:03.997226 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 16:30:03.997276 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"thread_start_us":151,"threads_started":1}
I20260809 16:30:03.997447 11733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.993165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45296 (local address 127.30.23.130:34291)
0809 16:30:03.993556 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:03.993560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:03.993726 (+ 166us) server_negotiation.cc:407] Connection header received
0809 16:30:03.993778 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:03.993780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:03.993796 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:03.993864 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:03.994038 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.994968 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.996100 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.996822 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.996890 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.996958 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:03.997251 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 16:30:03.997298 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":87,"threads_started":1}
I20260809 16:30:03.997613 11731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:03.992964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47362 (local address 127.30.23.129:45575)
0809 16:30:03.993159 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:03.993162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:03.993172 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:30:03.993234 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:03.993236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:03.993252 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:03.993387 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:03.993592 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.994339 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.995857 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:03.996443 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:03.996534 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:03.996617 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:03.997196 (+ 579us) server_negotiation.cc:299] Negotiation successful
0809 16:30:03.997244 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":100,"threads_started":1}
I20260809 16:30:03.998006 11579 tablet_service.cc:1467] Processing CreateTablet for tablet 85b0f116d87349e79049d948f5638bec (DEFAULT_TABLE table=test-table [id=433e040da5654a9398d2aa6d1d5c1a58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:03.998234 11066 tablet_service.cc:1467] Processing CreateTablet for tablet 85b0f116d87349e79049d948f5638bec (DEFAULT_TABLE table=test-table [id=433e040da5654a9398d2aa6d1d5c1a58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:03.998464 11579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b0f116d87349e79049d948f5638bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:03.998585 11066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b0f116d87349e79049d948f5638bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.001801 11321 tablet_service.cc:1467] Processing CreateTablet for tablet 85b0f116d87349e79049d948f5638bec (DEFAULT_TABLE table=test-table [id=433e040da5654a9398d2aa6d1d5c1a58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:04.002177 11321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b0f116d87349e79049d948f5638bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.002542 11734 tablet_bootstrap.cc:492] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf: Bootstrap starting.
I20260809 16:30:04.002911 11735 tablet_bootstrap.cc:492] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f: Bootstrap starting.
I20260809 16:30:04.003379 11734 tablet_bootstrap.cc:654] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.003553 11735 tablet_bootstrap.cc:654] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.003739 11734 log.cc:824] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf: Log is configured to *not* fsync() on all Append() calls
I20260809 16:30:04.003966 11735 log.cc:824] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:30:04.004599 11734 tablet_bootstrap.cc:492] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf: No bootstrap required, opened a new log
I20260809 16:30:04.004669 11735 tablet_bootstrap.cc:492] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f: No bootstrap required, opened a new log
I20260809 16:30:04.004683 11734 ts_tablet_manager.cc:1397] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:30:04.004751 11735 ts_tablet_manager.cc:1397] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:30:04.006414 11741 tablet_bootstrap.cc:492] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd: Bootstrap starting.
I20260809 16:30:04.007087 11741 tablet_bootstrap.cc:654] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.007175 11734 raft_consensus.cc:348] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } }
I20260809 16:30:04.007184 11735 raft_consensus.cc:348] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } }
I20260809 16:30:04.007366 11735 raft_consensus.cc:374] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.007380 11734 raft_consensus.cc:374] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.007426 11735 raft_consensus.cc:729] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba6ed2ba52344f3b20cba56e29f8f4f, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.007426 11734 raft_consensus.cc:729] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ba16574fc54f1180edcab741b6d4bf, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.007464 11741 log.cc:824] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd: Log is configured to *not* fsync() on all Append() calls
I20260809 16:30:04.007575 11735 consensus_queue.cc:260] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } }
I20260809 16:30:04.007575 11734 consensus_queue.cc:260] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } }
I20260809 16:30:04.007880 11735 ts_tablet_manager.cc:1428] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:30:04.007926 11192 heartbeater.cc:503] Master 127.30.23.190:44729 was elected leader, sending a full tablet report...
I20260809 16:30:04.007997 11687 heartbeater.cc:503] Master 127.30.23.190:44729 was elected leader, sending a full tablet report...
I20260809 16:30:04.008190 11734 ts_tablet_manager.cc:1428] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 16:30:04.008584 11741 tablet_bootstrap.cc:492] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd: No bootstrap required, opened a new log
I20260809 16:30:04.008662 11741 ts_tablet_manager.cc:1397] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:30:04.010344 11741 raft_consensus.cc:348] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } }
I20260809 16:30:04.010491 11741 raft_consensus.cc:374] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.010515 11741 raft_consensus.cc:729] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b60fe0d3864708862a8c5256a586fd, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.010632 11741 consensus_queue.cc:260] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } }
I20260809 16:30:04.010880 11741 ts_tablet_manager.cc:1428] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:30:04.010958 11447 heartbeater.cc:503] Master 127.30.23.190:44729 was elected leader, sending a full tablet report...
W20260809 16:30:04.023792 11449 tablet.cc:2369] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:30:04.077214 11746 raft_consensus.cc:482] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:30:04.077409 11746 raft_consensus.cc:504] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } }
I20260809 16:30:04.077857 11746 leader_election.cc:288] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131:36445), eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129:45575)
I20260809 16:30:04.082108 11731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.078118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:41915 (local address 127.30.23.129:45575)
0809 16:30:04.078174 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.078177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.078190 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:30:04.078256 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.078258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.078268 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.078340 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:04.078508 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.079304 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.081060 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.081642 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.081725 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.081735 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.082012 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.082050 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:30:04.082185 11246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.078062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45575 (local address 127.30.23.130:41915)
0809 16:30:04.078124 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:04.078136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:04.078185 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:04.078346 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:04.078347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:04.078349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:04.078437 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.078439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.079451 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.079452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.080939 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.080943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.081737 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.081738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.081908 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.081912 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:30:04.081984 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:30:04.082031 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":95}
I20260809 16:30:04.082602 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.078085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:36445 (local address 127.30.23.130:49733)
0809 16:30:04.078330 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:04.078340 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:04.078395 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:04.078549 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:04.078550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:04.078551 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:04.078615 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.078616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.079473 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.079474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.080939 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.080943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.081737 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.081738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.082332 (+ 594us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.082337 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:30:04.082457 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 16:30:04.082502 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":58,"thread_start_us":137,"threads_started":1}
I20260809 16:30:04.082643 11730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.078170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:49733 (local address 127.30.23.131:36445)
0809 16:30:04.078227 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.078229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.078406 (+ 177us) server_negotiation.cc:407] Connection header received
0809 16:30:04.078448 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.078449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.078456 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.078529 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:04.078680 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.079385 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.081061 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.081652 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.081726 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.081735 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.082556 (+ 821us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.082597 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:30:04.082602 11109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b0f116d87349e79049d948f5638bec" candidate_uuid: "a9b60fe0d3864708862a8c5256a586fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" is_pre_election: true
I20260809 16:30:04.082811 11109 raft_consensus.cc:2454] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b60fe0d3864708862a8c5256a586fd in term 0.
I20260809 16:30:04.083079 11240 leader_election.cc:302] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9b60fe0d3864708862a8c5256a586fd, eba6ed2ba52344f3b20cba56e29f8f4f; no voters:
I20260809 16:30:04.083184 11616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b0f116d87349e79049d948f5638bec" candidate_uuid: "a9b60fe0d3864708862a8c5256a586fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ba16574fc54f1180edcab741b6d4bf" is_pre_election: true
I20260809 16:30:04.083220 11746 raft_consensus.cc:2781] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:30:04.083293 11746 raft_consensus.cc:482] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:30:04.083336 11746 raft_consensus.cc:3037] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.083364 11616 raft_consensus.cc:2454] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b60fe0d3864708862a8c5256a586fd in term 0.
I20260809 16:30:04.084163 11746 raft_consensus.cc:504] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } }
I20260809 16:30:04.084328 11746 leader_election.cc:288] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [CANDIDATE]: Term 1 election: Requested vote from peers 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131:36445), eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129:45575)
I20260809 16:30:04.084489 11616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b0f116d87349e79049d948f5638bec" candidate_uuid: "a9b60fe0d3864708862a8c5256a586fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ba16574fc54f1180edcab741b6d4bf"
I20260809 16:30:04.084559 11616 raft_consensus.cc:3037] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.085429 11109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b0f116d87349e79049d948f5638bec" candidate_uuid: "a9b60fe0d3864708862a8c5256a586fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f"
I20260809 16:30:04.085508 11109 raft_consensus.cc:3037] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.085713 11616 raft_consensus.cc:2454] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b60fe0d3864708862a8c5256a586fd in term 1.
I20260809 16:30:04.085954 11233 leader_election.cc:302] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ba16574fc54f1180edcab741b6d4bf, a9b60fe0d3864708862a8c5256a586fd; no voters:
I20260809 16:30:04.086063 11746 raft_consensus.cc:2781] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:30:04.086247 11746 raft_consensus.cc:686] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [term 1 LEADER]: Becoming Leader. State: Replica: a9b60fe0d3864708862a8c5256a586fd, State: Running, Role: LEADER
I20260809 16:30:04.086371 11746 consensus_queue.cc:237] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } }
I20260809 16:30:04.086597 11109 raft_consensus.cc:2454] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b60fe0d3864708862a8c5256a586fd in term 1.
I20260809 16:30:04.087478 10837 catalog_manager.cc:5495] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd reported cstate change: term changed from 0 to 1, leader changed from <none> to a9b60fe0d3864708862a8c5256a586fd (127.30.23.130). New cstate: current_term: 1 leader_uuid: "a9b60fe0d3864708862a8c5256a586fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } health_report { overall_health: HEALTHY } } }
W20260809 16:30:04.098672 11195 tablet.cc:2369] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:30:04.113687 11733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.111099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45306 (local address 127.30.23.130:34291)
0809 16:30:04.111160 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.111162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.111184 (+ 22us) server_negotiation.cc:407] Connection header received
0809 16:30:04.111231 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.111232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.111249 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.111326 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:30:04.111505 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.111984 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.112731 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.113014 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.113094 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.113277 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:30:04.113453 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:30:04.113556 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.113575 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.113599 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:30:04.115665 11109 raft_consensus.cc:1261] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Refusing update from remote peer a9b60fe0d3864708862a8c5256a586fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:04.115706 11616 raft_consensus.cc:1261] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Refusing update from remote peer a9b60fe0d3864708862a8c5256a586fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:04.116052 11749 consensus_queue.cc:1035] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:04.116174 11746 consensus_queue.cc:1035] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:04.118443 11756 mvcc.cc:205] Tried to move back new op lower bound from 7316656144854822912 to 7316656144738893824. Current Snapshot: MvccSnapshot[applied={T|T < 7316656144854822912}]
I20260809 16:30:04.118696 11759 mvcc.cc:205] Tried to move back new op lower bound from 7316656144854822912 to 7316656144738893824. Current Snapshot: MvccSnapshot[applied={T|T < 7316656144854822912}]
I20260809 16:30:04.118925 11761 mvcc.cc:205] Tried to move back new op lower bound from 7316656144854822912 to 7316656144738893824. Current Snapshot: MvccSnapshot[applied={T|T < 7316656144854822912}]
I20260809 16:30:04.121479 10837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:30:04.121592 10837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:30:04.122467 10837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7: Sending DeleteTablet for 3 replicas of tablet 85b0f116d87349e79049d948f5638bec
I20260809 16:30:04.122835 11066 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b0f116d87349e79049d948f5638bec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:04 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:47362
I20260809 16:30:04.122861 11579 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b0f116d87349e79049d948f5638bec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:04 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:55160
I20260809 16:30:04.123054 11321 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b0f116d87349e79049d948f5638bec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:04 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:45296
I20260809 16:30:04.123143 10837 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:30:04.123302 10837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:30:04.123353 11784 tablet_replica.cc:330] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd: stopping tablet replica
I20260809 16:30:04.123353 11782 tablet_replica.cc:330] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf: stopping tablet replica
I20260809 16:30:04.123447 11784 raft_consensus.cc:2229] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:30:04.123468 11782 raft_consensus.cc:2229] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:30:04.123575 11784 raft_consensus.cc:2258] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:04.123585 11782 raft_consensus.cc:2258] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:04.124090 11784 ts_tablet_manager.cc:1905] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:04.124102 11782 ts_tablet_manager.cc:1905] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:04.125262 11066 tablet_service.cc:1467] Processing CreateTablet for tablet cb361d0b72d644cd9a33cc3a960317d4 (DEFAULT_TABLE table=test-table [id=9c511faeaf0d442893655704ef7bd4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:04.125422 11066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb361d0b72d644cd9a33cc3a960317d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.125749 11321 tablet_service.cc:1467] Processing CreateTablet for tablet cb361d0b72d644cd9a33cc3a960317d4 (DEFAULT_TABLE table=test-table [id=9c511faeaf0d442893655704ef7bd4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:04.125883 11782 ts_tablet_manager.cc:1918] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:30:04.125967 11321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb361d0b72d644cd9a33cc3a960317d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.126040 11782 log.cc:1196] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/85b0f116d87349e79049d948f5638bec
I20260809 16:30:04.126055 11780 tablet_replica.cc:330] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f: stopping tablet replica
I20260809 16:30:04.126163 11780 raft_consensus.cc:2229] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:30:04.126272 11780 raft_consensus.cc:2258] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:04.126353 11782 ts_tablet_manager.cc:1939] T 85b0f116d87349e79049d948f5638bec P 44ba16574fc54f1180edcab741b6d4bf: Deleting consensus metadata
I20260809 16:30:04.125968 11784 ts_tablet_manager.cc:1918] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:30:04.126449 11784 log.cc:1196] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/85b0f116d87349e79049d948f5638bec
I20260809 16:30:04.126719 11784 ts_tablet_manager.cc:1939] T 85b0f116d87349e79049d948f5638bec P a9b60fe0d3864708862a8c5256a586fd: Deleting consensus metadata
I20260809 16:30:04.126803 11780 ts_tablet_manager.cc:1905] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:04.127290 11741 tablet_bootstrap.cc:492] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd: Bootstrap starting.
I20260809 16:30:04.127435 10811 catalog_manager.cc:4841] TS a9b60fe0d3864708862a8c5256a586fd (127.30.23.130:34291): tablet 85b0f116d87349e79049d948f5638bec (table test-table [id=433e040da5654a9398d2aa6d1d5c1a58]) successfully deleted
I20260809 16:30:04.127746 11741 tablet_bootstrap.cc:654] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.128345 11741 tablet_bootstrap.cc:492] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd: No bootstrap required, opened a new log
I20260809 16:30:04.128392 11741 ts_tablet_manager.cc:1397] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:04.128551 11741 raft_consensus.cc:348] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.128654 11741 raft_consensus.cc:374] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.128675 11741 raft_consensus.cc:729] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b60fe0d3864708862a8c5256a586fd, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.128720 11741 consensus_queue.cc:260] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.128801 11741 ts_tablet_manager.cc:1428] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:30:04.129112 11735 tablet_bootstrap.cc:492] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f: Bootstrap starting.
I20260809 16:30:04.129540 11735 tablet_bootstrap.cc:654] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.129777 11780 ts_tablet_manager.cc:1918] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:30:04.129906 11780 log.cc:1196] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/wal/wals/85b0f116d87349e79049d948f5638bec
I20260809 16:30:04.129823 10810 catalog_manager.cc:4841] TS 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131:36445): tablet 85b0f116d87349e79049d948f5638bec (table test-table [id=433e040da5654a9398d2aa6d1d5c1a58]) successfully deleted
I20260809 16:30:04.130194 11780 ts_tablet_manager.cc:1939] T 85b0f116d87349e79049d948f5638bec P eba6ed2ba52344f3b20cba56e29f8f4f: Deleting consensus metadata
I20260809 16:30:04.130563 11579 tablet_service.cc:1467] Processing CreateTablet for tablet cb361d0b72d644cd9a33cc3a960317d4 (DEFAULT_TABLE table=test-table [id=9c511faeaf0d442893655704ef7bd4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:04.130703 11579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb361d0b72d644cd9a33cc3a960317d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.130815 10813 catalog_manager.cc:4841] TS eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129:45575): tablet 85b0f116d87349e79049d948f5638bec (table test-table [id=433e040da5654a9398d2aa6d1d5c1a58]) successfully deleted
I20260809 16:30:04.130980 11735 tablet_bootstrap.cc:492] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f: No bootstrap required, opened a new log
I20260809 16:30:04.131052 11735 ts_tablet_manager.cc:1397] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:30:04.131348 11735 raft_consensus.cc:348] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.131434 11735 raft_consensus.cc:374] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.131455 11735 raft_consensus.cc:729] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba6ed2ba52344f3b20cba56e29f8f4f, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.131508 11735 consensus_queue.cc:260] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.131599 11735 ts_tablet_manager.cc:1428] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:30:04.132468 11734 tablet_bootstrap.cc:492] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf: Bootstrap starting.
I20260809 16:30:04.132903 11734 tablet_bootstrap.cc:654] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.133670 11734 tablet_bootstrap.cc:492] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf: No bootstrap required, opened a new log
I20260809 16:30:04.133728 11734 ts_tablet_manager.cc:1397] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:04.133878 11734 raft_consensus.cc:348] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.133960 11734 raft_consensus.cc:374] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.133981 11734 raft_consensus.cc:729] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ba16574fc54f1180edcab741b6d4bf, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.134027 11734 consensus_queue.cc:260] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.134119 11734 ts_tablet_manager.cc:1428] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:30:04.176751 11769 raft_consensus.cc:482] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:30:04.176880 11769 raft_consensus.cc:504] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.177083 11769 leader_election.cc:288] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129:45575), 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131:36445)
I20260809 16:30:04.177414 11109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb361d0b72d644cd9a33cc3a960317d4" candidate_uuid: "a9b60fe0d3864708862a8c5256a586fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" is_pre_election: true
I20260809 16:30:04.177416 11616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb361d0b72d644cd9a33cc3a960317d4" candidate_uuid: "a9b60fe0d3864708862a8c5256a586fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ba16574fc54f1180edcab741b6d4bf" is_pre_election: true
I20260809 16:30:04.177523 11109 raft_consensus.cc:2454] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b60fe0d3864708862a8c5256a586fd in term 0.
I20260809 16:30:04.177548 11616 raft_consensus.cc:2454] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b60fe0d3864708862a8c5256a586fd in term 0.
I20260809 16:30:04.177732 11240 leader_election.cc:302] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9b60fe0d3864708862a8c5256a586fd, eba6ed2ba52344f3b20cba56e29f8f4f; no voters:
I20260809 16:30:04.177870 11769 raft_consensus.cc:2781] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:30:04.177906 11769 raft_consensus.cc:482] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:30:04.177934 11769 raft_consensus.cc:3037] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.178829 11769 raft_consensus.cc:504] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.178994 11769 leader_election.cc:288] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [CANDIDATE]: Term 1 election: Requested vote from peers eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129:45575), 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131:36445)
I20260809 16:30:04.179207 11109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb361d0b72d644cd9a33cc3a960317d4" candidate_uuid: "a9b60fe0d3864708862a8c5256a586fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f"
I20260809 16:30:04.179266 11616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb361d0b72d644cd9a33cc3a960317d4" candidate_uuid: "a9b60fe0d3864708862a8c5256a586fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ba16574fc54f1180edcab741b6d4bf"
I20260809 16:30:04.179316 11109 raft_consensus.cc:3037] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.179366 11616 raft_consensus.cc:3037] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.179994 11616 raft_consensus.cc:2454] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b60fe0d3864708862a8c5256a586fd in term 1.
I20260809 16:30:04.180119 11109 raft_consensus.cc:2454] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b60fe0d3864708862a8c5256a586fd in term 1.
I20260809 16:30:04.180171 11233 leader_election.cc:302] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ba16574fc54f1180edcab741b6d4bf, a9b60fe0d3864708862a8c5256a586fd; no voters:
I20260809 16:30:04.180289 11769 raft_consensus.cc:2781] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:30:04.180365 11769 raft_consensus.cc:686] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [term 1 LEADER]: Becoming Leader. State: Replica: a9b60fe0d3864708862a8c5256a586fd, State: Running, Role: LEADER
I20260809 16:30:04.180421 11769 consensus_queue.cc:237] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.181051 10836 catalog_manager.cc:5495] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd reported cstate change: term changed from 0 to 1, leader changed from <none> to a9b60fe0d3864708862a8c5256a586fd (127.30.23.130). New cstate: current_term: 1 leader_uuid: "a9b60fe0d3864708862a8c5256a586fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } health_report { overall_health: UNKNOWN } } }
W20260809 16:30:04.196578 11688 tablet.cc:2369] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 16:30:04 dist-test-slave-j2vl krb5kdc[10760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293003, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.190@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:30:04.218951 11803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.210336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44729 (local address 127.0.0.1:39836)
0809 16:30:04.210516 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:04.210536 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:04.210601 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:04.210757 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:04.210758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:04.211067 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:04.211117 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.211119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.212139 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.212140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.212995 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.212998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.213503 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.213504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.213549 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.214181 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:04.214186 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:04.215874 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:04.217860 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:04.217868 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:04.217870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:04.218071 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:04.218300 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:04.218301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:04.218301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:04.218370 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:04.218663 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:04.218670 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:04.218712 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 16:30:04.218763 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:30:04.218874 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
I20260809 16:30:04.219280 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.210480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39836 (local address 127.30.23.190:44729)
0809 16:30:04.210533 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.210535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.210610 (+ 75us) server_negotiation.cc:407] Connection header received
0809 16:30:04.210645 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.210646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.210659 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.210745 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:04.211216 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.212008 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.213093 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.213422 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.213503 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.215973 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:04.215975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:04.215975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:04.215983 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:04.217775 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:04.218157 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:04.218158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:04.218159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:04.218227 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:04.218440 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:04.218441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:04.218442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:04.218594 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:04.218672 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.218818 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.218955 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":30}
I20260809 16:30:04.221722 10836 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:39836:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 16:30:04.221964 10836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:30:04.223635 11066 tablet_service.cc:1467] Processing CreateTablet for tablet 3bb4f0e1ee4e40e49c824c123a72c0e9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=25eafa14a3d84ac78bbd24adb28e1080]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:30:04.223717 11321 tablet_service.cc:1467] Processing CreateTablet for tablet 3bb4f0e1ee4e40e49c824c123a72c0e9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=25eafa14a3d84ac78bbd24adb28e1080]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:30:04.223842 11321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bb4f0e1ee4e40e49c824c123a72c0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.223839 11066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bb4f0e1ee4e40e49c824c123a72c0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.225504 11579 tablet_service.cc:1467] Processing CreateTablet for tablet 3bb4f0e1ee4e40e49c824c123a72c0e9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=25eafa14a3d84ac78bbd24adb28e1080]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:30:04.225672 11579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bb4f0e1ee4e40e49c824c123a72c0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.225689 11735 tablet_bootstrap.cc:492] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f: Bootstrap starting.
I20260809 16:30:04.226178 11735 tablet_bootstrap.cc:654] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.226933 11735 tablet_bootstrap.cc:492] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f: No bootstrap required, opened a new log
I20260809 16:30:04.226990 11735 ts_tablet_manager.cc:1397] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:04.227136 11735 raft_consensus.cc:348] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.227223 11735 raft_consensus.cc:374] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.227252 11735 raft_consensus.cc:729] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba6ed2ba52344f3b20cba56e29f8f4f, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.227306 11735 consensus_queue.cc:260] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.227409 11735 ts_tablet_manager.cc:1428] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:04.227555 11734 tablet_bootstrap.cc:492] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf: Bootstrap starting.
I20260809 16:30:04.227640 11743 tablet_replica.cc:439] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } }
I20260809 16:30:04.227741 11743 tablet_replica.cc:442] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:30:04.228010 11734 tablet_bootstrap.cc:654] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.228502 11741 tablet_bootstrap.cc:492] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd: Bootstrap starting.
I20260809 16:30:04.228951 11741 tablet_bootstrap.cc:654] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.228986 11734 tablet_bootstrap.cc:492] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf: No bootstrap required, opened a new log
I20260809 16:30:04.229032 11734 ts_tablet_manager.cc:1397] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:04.229190 11734 raft_consensus.cc:348] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.229269 11734 raft_consensus.cc:374] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.229288 11734 raft_consensus.cc:729] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ba16574fc54f1180edcab741b6d4bf, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.229398 11734 consensus_queue.cc:260] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.229518 11734 ts_tablet_manager.cc:1428] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:30:04.229676 11744 tablet_replica.cc:439] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } }
I20260809 16:30:04.229704 11741 tablet_bootstrap.cc:492] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd: No bootstrap required, opened a new log
I20260809 16:30:04.229756 11744 tablet_replica.cc:442] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:30:04.229785 11741 ts_tablet_manager.cc:1397] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:04.229980 11741 raft_consensus.cc:348] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.230072 11741 raft_consensus.cc:374] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.230091 11741 raft_consensus.cc:729] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b60fe0d3864708862a8c5256a586fd, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.230137 11741 consensus_queue.cc:260] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.230223 11741 ts_tablet_manager.cc:1428] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:04.230244 11769 tablet_replica.cc:439] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } }
I20260809 16:30:04.230348 11769 tablet_replica.cc:442] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:30:04.361538 11743 raft_consensus.cc:482] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:30:04.361644 11743 raft_consensus.cc:504] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.362040 11743 leader_election.cc:288] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9b60fe0d3864708862a8c5256a586fd (127.30.23.130:34291), 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131:36445)
I20260809 16:30:04.366458 11730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.362481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:44529 (local address 127.30.23.131:36445)
0809 16:30:04.362557 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.362559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.362594 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:30:04.362640 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.362641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.362652 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.362710 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:04.362983 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.363776 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.365464 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.366001 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.366083 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.366093 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.366349 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.366389 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 16:30:04.366458 11733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.362322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:51539 (local address 127.30.23.130:34291)
0809 16:30:04.362380 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.362383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.362554 (+ 171us) server_negotiation.cc:407] Connection header received
0809 16:30:04.362664 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.362666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.362680 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.362739 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:04.362991 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.363773 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.365454 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.366026 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.366092 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.366100 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.366344 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.366385 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:30:04.366504 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.362271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:34291 (local address 127.30.23.129:51539)
0809 16:30:04.362490 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:04.362499 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:04.362580 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:04.362762 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:04.362763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:04.362765 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:04.362891 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.362894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.363884 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.363885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.365367 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.365369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.366124 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.366124 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.366230 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.366238 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 16:30:04.366326 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:30:04.366385 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":107,"thread_start_us":144,"threads_started":1}
I20260809 16:30:04.366717 11810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.362246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:36445 (local address 127.30.23.129:44529)
0809 16:30:04.362475 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:04.362488 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:04.362556 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:04.362769 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:04.362769 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:04.362771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:04.362891 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.362894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.363884 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.363885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.365361 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.365365 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.366123 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.366124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.366230 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.366240 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 16:30:04.366326 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:30:04.366385 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":91,"thread_start_us":124,"threads_started":1}
I20260809 16:30:04.367015 11616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb4f0e1ee4e40e49c824c123a72c0e9" candidate_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ba16574fc54f1180edcab741b6d4bf" is_pre_election: true
I20260809 16:30:04.366988 11352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb4f0e1ee4e40e49c824c123a72c0e9" candidate_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9b60fe0d3864708862a8c5256a586fd" is_pre_election: true
I20260809 16:30:04.367113 11616 raft_consensus.cc:2454] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eba6ed2ba52344f3b20cba56e29f8f4f in term 0.
I20260809 16:30:04.367128 11352 raft_consensus.cc:2454] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eba6ed2ba52344f3b20cba56e29f8f4f in term 0.
I20260809 16:30:04.367305 10981 leader_election.cc:302] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ba16574fc54f1180edcab741b6d4bf, eba6ed2ba52344f3b20cba56e29f8f4f; no voters:
I20260809 16:30:04.367477 11743 raft_consensus.cc:2781] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:30:04.367538 11743 raft_consensus.cc:482] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:30:04.367554 11743 raft_consensus.cc:3037] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.368363 11743 raft_consensus.cc:504] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.368517 11743 leader_election.cc:288] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [CANDIDATE]: Term 1 election: Requested vote from peers a9b60fe0d3864708862a8c5256a586fd (127.30.23.130:34291), 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131:36445)
I20260809 16:30:04.368731 11352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb4f0e1ee4e40e49c824c123a72c0e9" candidate_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9b60fe0d3864708862a8c5256a586fd"
I20260809 16:30:04.368768 11616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb4f0e1ee4e40e49c824c123a72c0e9" candidate_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ba16574fc54f1180edcab741b6d4bf"
I20260809 16:30:04.368842 11352 raft_consensus.cc:3037] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.368845 11616 raft_consensus.cc:3037] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.369544 11616 raft_consensus.cc:2454] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eba6ed2ba52344f3b20cba56e29f8f4f in term 1.
I20260809 16:30:04.369715 10981 leader_election.cc:302] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ba16574fc54f1180edcab741b6d4bf, eba6ed2ba52344f3b20cba56e29f8f4f; no voters:
I20260809 16:30:04.369819 11743 raft_consensus.cc:2781] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:30:04.369813 11352 raft_consensus.cc:2454] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eba6ed2ba52344f3b20cba56e29f8f4f in term 1.
I20260809 16:30:04.370035 11743 raft_consensus.cc:686] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 LEADER]: Becoming Leader. State: Replica: eba6ed2ba52344f3b20cba56e29f8f4f, State: Running, Role: LEADER
I20260809 16:30:04.370115 11743 consensus_queue.cc:237] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.370527 11812 tablet_replica.cc:439] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f: TxnStatusTablet state changed. Reason: New leader eba6ed2ba52344f3b20cba56e29f8f4f. Latest consensus state: current_term: 1 leader_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } }
I20260809 16:30:04.370639 11812 tablet_replica.cc:442] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:30:04.370838 11814 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 16:30:04.370975 11814 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 16:30:04.371008 10836 catalog_manager.cc:5495] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f reported cstate change: term changed from 0 to 1, leader changed from <none> to eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129). New cstate: current_term: 1 leader_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } health_report { overall_health: UNKNOWN } } }
I20260809 16:30:04.379554 11731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.376380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47376 (local address 127.30.23.129:45575)
0809 16:30:04.376430 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.376432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.376539 (+ 107us) server_negotiation.cc:407] Connection header received
0809 16:30:04.376612 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.376613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.376630 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.376695 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:30:04.376867 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.377450 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.378669 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.378869 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.378955 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.379173 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:30:04.379320 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:30:04.379385 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.379471 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.379491 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:30:04.379612 11803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.376344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45575 (local address 127.0.0.1:47376)
0809 16:30:04.376440 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:04.376455 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:04.376527 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:04.376719 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:04.376720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:04.376721 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:30:04.376780 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.376784 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.377532 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.377534 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.378507 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.378511 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.378966 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.378967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.379043 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.379066 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:30:04.379395 (+ 329us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:30:04.379406 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 16:30:04.379467 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 16:30:04.379526 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 16:30:04.381073 11616 raft_consensus.cc:1261] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Refusing update from remote peer eba6ed2ba52344f3b20cba56e29f8f4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:04.381229 11352 raft_consensus.cc:1261] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Refusing update from remote peer eba6ed2ba52344f3b20cba56e29f8f4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:04.381361 11812 consensus_queue.cc:1035] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:04.381520 11743 consensus_queue.cc:1035] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:04.382063 11744 tablet_replica.cc:439] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf: TxnStatusTablet state changed. Reason: New leader eba6ed2ba52344f3b20cba56e29f8f4f. Latest consensus state: current_term: 1 leader_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } }
I20260809 16:30:04.382180 11744 tablet_replica.cc:442] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:30:04.382184 11769 tablet_replica.cc:439] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd: TxnStatusTablet state changed. Reason: New leader eba6ed2ba52344f3b20cba56e29f8f4f. Latest consensus state: current_term: 1 leader_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } }
I20260809 16:30:04.382262 11769 tablet_replica.cc:442] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:30:04.383113 11812 tablet_replica.cc:439] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } }
I20260809 16:30:04.383209 11812 tablet_replica.cc:442] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:30:04.383821 11744 tablet_replica.cc:439] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } }
I20260809 16:30:04.383919 11744 tablet_replica.cc:442] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P 44ba16574fc54f1180edcab741b6d4bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:30:04.384351 11743 tablet_replica.cc:439] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } }
I20260809 16:30:04.384450 11743 tablet_replica.cc:442] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P eba6ed2ba52344f3b20cba56e29f8f4f: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:30:04.384910 11769 tablet_replica.cc:439] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } } }
I20260809 16:30:04.385010 11769 tablet_replica.cc:442] T 3bb4f0e1ee4e40e49c824c123a72c0e9 P a9b60fe0d3864708862a8c5256a586fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:30:04.390115 11731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.387344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:54899 (local address 127.30.23.129:45575)
0809 16:30:04.387409 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.387411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.387447 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:30:04.387513 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.387514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.387524 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.387586 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:30:04.387676 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.388418 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.389191 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.389525 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.389599 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.389809 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:30:04.389848 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:30:04.389910 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.390016 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.390040 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:30:04.390141 11824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.387258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45575 (local address 127.30.23.130:54899)
0809 16:30:04.387388 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:04.387400 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:04.387469 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:04.387588 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:04.387588 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:04.387590 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:30:04.387634 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.387635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.388501 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.388502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.389108 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.389110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.389619 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.389620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.389674 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.389690 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:30:04.389959 (+ 269us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:30:04.389960 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:30:04.390008 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 16:30:04.390066 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":66,"threads_started":1}
I20260809 16:30:04.392479 11616 raft_consensus.cc:1261] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Refusing update from remote peer a9b60fe0d3864708862a8c5256a586fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:04.392479 11109 raft_consensus.cc:1261] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Refusing update from remote peer a9b60fe0d3864708862a8c5256a586fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:04.392685 11769 consensus_queue.cc:1035] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:04.392787 11746 consensus_queue.cc:1035] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:04.402132 11831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.398862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:34291 (local address 127.30.23.129:40873)
0809 16:30:04.399037 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:04.399048 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:04.399111 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:04.399240 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:04.399241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:04.399242 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:30:04.399293 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.399294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.400187 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.400188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.401154 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.401157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.401619 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.401621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.401693 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.401712 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:30:04.401922 (+ 210us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:30:04.401924 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:30:04.401974 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:30:04.402018 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":106,"threads_started":1}
I20260809 16:30:04.402117 11733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.398960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:40873 (local address 127.30.23.130:34291)
0809 16:30:04.399003 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.399005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.399120 (+ 115us) server_negotiation.cc:407] Connection header received
0809 16:30:04.399161 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.399162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.399172 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.399217 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:30:04.399348 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.400092 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.401248 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.401518 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.401611 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.401804 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:30:04.401857 (+ 53us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:30:04.401919 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.401980 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.401999 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:30:04.410382 10836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:30:04.410499 10836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:30:04.411192 10836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7: Sending DeleteTablet for 3 replicas of tablet cb361d0b72d644cd9a33cc3a960317d4
I20260809 16:30:04.411471 11579 tablet_service.cc:1514] Processing DeleteTablet for tablet cb361d0b72d644cd9a33cc3a960317d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:04 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:55160
I20260809 16:30:04.411484 11066 tablet_service.cc:1514] Processing DeleteTablet for tablet cb361d0b72d644cd9a33cc3a960317d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:04 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:47362
I20260809 16:30:04.411569 11782 tablet_replica.cc:330] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf: stopping tablet replica
I20260809 16:30:04.411589 11780 tablet_replica.cc:330] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f: stopping tablet replica
I20260809 16:30:04.411621 11782 raft_consensus.cc:2229] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:30:04.411638 11780 raft_consensus.cc:2229] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:30:04.411657 11782 pending_rounds.cc:62] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf: Trying to abort 1 pending ops.
I20260809 16:30:04.411674 11780 raft_consensus.cc:2258] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:04.411675 11782 pending_rounds.cc:68] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf: Aborting op as it isn't in flight: id { term: 1 index: 5 } timestamp: 7316656146044772352 op_type: PARTICIPANT_OP participant_request { tablet_id: "cb361d0b72d644cd9a33cc3a960317d4" op { type: FINALIZE_COMMIT txn_id: 0 finalized_commit_timestamp: 7316656146035056640 } }
I20260809 16:30:04.411813 11782 raft_consensus.cc:2258] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:04.412204 11780 ts_tablet_manager.cc:1905] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 16:30:04.412252 11825 consensus_queue.cc:1159] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "44ba16574fc54f1180edcab741b6d4bf" status { last_received { term: 1 index: 5 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 5 } }
W20260809 16:30:04.412578 11782 mvcc.cc:119] aborting op with timestamp 7316656146035056640 in state 0; MVCC is closed
I20260809 16:30:04.412653 11782 ts_tablet_manager.cc:1905] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:04.413770 11321 tablet_service.cc:1514] Processing DeleteTablet for tablet cb361d0b72d644cd9a33cc3a960317d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:04 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:45296
I20260809 16:30:04.413863 11784 tablet_replica.cc:330] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd: stopping tablet replica
I20260809 16:30:04.413915 11784 raft_consensus.cc:2229] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:30:04.413969 11784 raft_consensus.cc:2258] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:04.414142 11780 ts_tablet_manager.cc:1918] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:30:04.414201 11780 log.cc:1196] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/wal/wals/cb361d0b72d644cd9a33cc3a960317d4
I20260809 16:30:04.414256 11784 ts_tablet_manager.cc:1905] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:04.414425 11780 ts_tablet_manager.cc:1939] T cb361d0b72d644cd9a33cc3a960317d4 P eba6ed2ba52344f3b20cba56e29f8f4f: Deleting consensus metadata
I20260809 16:30:04.414515 11782 ts_tablet_manager.cc:1918] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:30:04.414565 11782 log.cc:1196] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/cb361d0b72d644cd9a33cc3a960317d4
I20260809 16:30:04.414786 11782 ts_tablet_manager.cc:1939] T cb361d0b72d644cd9a33cc3a960317d4 P 44ba16574fc54f1180edcab741b6d4bf: Deleting consensus metadata
I20260809 16:30:04.414891 10813 catalog_manager.cc:4841] TS eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129:45575): tablet cb361d0b72d644cd9a33cc3a960317d4 (table test-table [id=9c511faeaf0d442893655704ef7bd4d7]) successfully deleted
I20260809 16:30:04.415174 10810 catalog_manager.cc:4841] TS 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131:36445): tablet cb361d0b72d644cd9a33cc3a960317d4 (table test-table [id=9c511faeaf0d442893655704ef7bd4d7]) successfully deleted
I20260809 16:30:04.415844 11784 ts_tablet_manager.cc:1918] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:30:04.415912 11784 log.cc:1196] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/cb361d0b72d644cd9a33cc3a960317d4
I20260809 16:30:04.416134 11784 ts_tablet_manager.cc:1939] T cb361d0b72d644cd9a33cc3a960317d4 P a9b60fe0d3864708862a8c5256a586fd: Deleting consensus metadata
I20260809 16:30:04.417797 10811 catalog_manager.cc:4841] TS a9b60fe0d3864708862a8c5256a586fd (127.30.23.130:34291): tablet cb361d0b72d644cd9a33cc3a960317d4 (table test-table [id=9c511faeaf0d442893655704ef7bd4d7]) successfully deleted
Aug 09 16:30:04 dist-test-slave-j2vl krb5kdc[10760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:30:04 dist-test-slave-j2vl krb5kdc[10760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:04 dist-test-slave-j2vl krb5kdc[10760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 16:30:04.439190 10932 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:04.433203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.30.23.190:44729)
0809 16:30:04.433281 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.433284 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.433416 (+ 132us) server_negotiation.cc:407] Connection header received
0809 16:30:04.433519 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.433521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.433534 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.433606 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:04.434221 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.435047 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.435783 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.436103 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.436180 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.439116 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39838: BlockingRecv error: recv got EOF from 127.0.0.1:39838 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 16:30:04 dist-test-slave-j2vl krb5kdc[10760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM
I20260809 16:30:04.447597 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.440021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.30.23.190:44729)
0809 16:30:04.440081 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.440083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.440094 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:30:04.440169 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.440170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.440181 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.440240 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:04.440691 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.441494 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.442157 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.442458 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.442531 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.444835 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:04.444837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:04.444837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:04.444845 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:04.446376 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:04.446703 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:04.446704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:04.446705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:04.446766 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:04.447022 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:04.447023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:04.447024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:04.447174 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:04.447282 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.447414 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.447498 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:30:04.449271 10836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:30:04.449522 10836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:30:04.451067 11321 tablet_service.cc:1467] Processing CreateTablet for tablet 7400bc6fd9794a3c948cb2313d4814bb (DEFAULT_TABLE table=test-table [id=297f4d8e2a5742a68ddf84fb093f54ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:04.451092 11066 tablet_service.cc:1467] Processing CreateTablet for tablet 7400bc6fd9794a3c948cb2313d4814bb (DEFAULT_TABLE table=test-table [id=297f4d8e2a5742a68ddf84fb093f54ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:04.451094 11579 tablet_service.cc:1467] Processing CreateTablet for tablet 7400bc6fd9794a3c948cb2313d4814bb (DEFAULT_TABLE table=test-table [id=297f4d8e2a5742a68ddf84fb093f54ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:04.451220 11321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7400bc6fd9794a3c948cb2313d4814bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.451220 11066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7400bc6fd9794a3c948cb2313d4814bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.451227 11579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7400bc6fd9794a3c948cb2313d4814bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.452551 11734 tablet_bootstrap.cc:492] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf: Bootstrap starting.
I20260809 16:30:04.452590 11741 tablet_bootstrap.cc:492] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd: Bootstrap starting.
I20260809 16:30:04.452950 11741 tablet_bootstrap.cc:654] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.452979 11734 tablet_bootstrap.cc:654] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.453281 11735 tablet_bootstrap.cc:492] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f: Bootstrap starting.
I20260809 16:30:04.453742 11735 tablet_bootstrap.cc:654] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.453758 11734 tablet_bootstrap.cc:492] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf: No bootstrap required, opened a new log
I20260809 16:30:04.453802 11734 ts_tablet_manager.cc:1397] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:30:04.453972 11734 raft_consensus.cc:348] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.454068 11734 raft_consensus.cc:374] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.454094 11734 raft_consensus.cc:729] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ba16574fc54f1180edcab741b6d4bf, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.454146 11734 consensus_queue.cc:260] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.454253 11734 ts_tablet_manager.cc:1428] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:30:04.454393 11741 tablet_bootstrap.cc:492] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd: No bootstrap required, opened a new log
I20260809 16:30:04.454445 11741 ts_tablet_manager.cc:1397] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:30:04.454571 11741 raft_consensus.cc:348] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.454648 11741 raft_consensus.cc:374] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.454663 11741 raft_consensus.cc:729] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b60fe0d3864708862a8c5256a586fd, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.454738 11741 consensus_queue.cc:260] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.454859 11741 ts_tablet_manager.cc:1428] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:30:04.455446 11735 tablet_bootstrap.cc:492] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f: No bootstrap required, opened a new log
I20260809 16:30:04.455502 11735 ts_tablet_manager.cc:1397] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:30:04.455608 11735 raft_consensus.cc:348] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.455674 11735 raft_consensus.cc:374] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.455689 11735 raft_consensus.cc:729] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba6ed2ba52344f3b20cba56e29f8f4f, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.455739 11735 consensus_queue.cc:260] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.455832 11735 ts_tablet_manager.cc:1428] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:04.500663 11744 raft_consensus.cc:482] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:30:04.500752 11744 raft_consensus.cc:504] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.501127 11744 leader_election.cc:288] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129:45575), a9b60fe0d3864708862a8c5256a586fd (127.30.23.130:34291)
I20260809 16:30:04.504908 11866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.501299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45575 (local address 127.30.23.131:46819)
0809 16:30:04.501468 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:04.501481 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:04.501544 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:04.501784 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:04.501785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:04.501787 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:04.501880 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.501881 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.502726 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.502727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.503922 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.503925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.504643 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.504644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.504715 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.504720 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:30:04.504776 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:30:04.504817 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260809 16:30:04.504947 11731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.501397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:46819 (local address 127.30.23.129:45575)
0809 16:30:04.501518 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.501520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.501531 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:30:04.501630 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.501632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.501644 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.501677 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:04.501968 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.502646 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.504006 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.504556 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.504640 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.504650 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.504800 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.504858 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 16:30:04.505475 11862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.501316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:34291 (local address 127.30.23.131:50531)
0809 16:30:04.501627 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:04.501637 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:04.501695 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:04.501834 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:04.501835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:04.501837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:04.501878 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.501881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.503568 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.503569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.504419 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:04.504421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.505184 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.505185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:04.505255 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.505259 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:30:04.505324 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 16:30:04.505386 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":82,"threads_started":1}
I20260809 16:30:04.505488 11733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.501538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:50531 (local address 127.30.23.130:34291)
0809 16:30:04.501589 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.501590 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.501709 (+ 119us) server_negotiation.cc:407] Connection header received
0809 16:30:04.501744 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.501745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.501759 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.501824 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:04.502769 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.503473 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.504503 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.505091 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.505161 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.505170 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.505371 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.505413 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:30:04.505753 11109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7400bc6fd9794a3c948cb2313d4814bb" candidate_uuid: "44ba16574fc54f1180edcab741b6d4bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" is_pre_election: true
I20260809 16:30:04.505774 11352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7400bc6fd9794a3c948cb2313d4814bb" candidate_uuid: "44ba16574fc54f1180edcab741b6d4bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9b60fe0d3864708862a8c5256a586fd" is_pre_election: true
I20260809 16:30:04.505858 11109 raft_consensus.cc:2454] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44ba16574fc54f1180edcab741b6d4bf in term 0.
I20260809 16:30:04.505870 11352 raft_consensus.cc:2454] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44ba16574fc54f1180edcab741b6d4bf in term 0.
I20260809 16:30:04.506042 11500 leader_election.cc:302] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ba16574fc54f1180edcab741b6d4bf, eba6ed2ba52344f3b20cba56e29f8f4f; no voters:
I20260809 16:30:04.506178 11744 raft_consensus.cc:2781] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:30:04.506218 11744 raft_consensus.cc:482] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:30:04.506238 11744 raft_consensus.cc:3037] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.507175 11744 raft_consensus.cc:504] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.507325 11744 leader_election.cc:288] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [CANDIDATE]: Term 1 election: Requested vote from peers eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129:45575), a9b60fe0d3864708862a8c5256a586fd (127.30.23.130:34291)
I20260809 16:30:04.507512 11352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7400bc6fd9794a3c948cb2313d4814bb" candidate_uuid: "44ba16574fc54f1180edcab741b6d4bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9b60fe0d3864708862a8c5256a586fd"
I20260809 16:30:04.507596 11352 raft_consensus.cc:3037] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.507951 11109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7400bc6fd9794a3c948cb2313d4814bb" candidate_uuid: "44ba16574fc54f1180edcab741b6d4bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f"
I20260809 16:30:04.508049 11109 raft_consensus.cc:3037] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.508378 11352 raft_consensus.cc:2454] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44ba16574fc54f1180edcab741b6d4bf in term 1.
I20260809 16:30:04.508581 11498 leader_election.cc:302] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ba16574fc54f1180edcab741b6d4bf, a9b60fe0d3864708862a8c5256a586fd; no voters:
I20260809 16:30:04.508675 11744 raft_consensus.cc:2781] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:30:04.508785 11744 raft_consensus.cc:686] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [term 1 LEADER]: Becoming Leader. State: Replica: 44ba16574fc54f1180edcab741b6d4bf, State: Running, Role: LEADER
I20260809 16:30:04.508778 11109 raft_consensus.cc:2454] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44ba16574fc54f1180edcab741b6d4bf in term 1.
I20260809 16:30:04.508854 11744 consensus_queue.cc:237] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.509600 10836 catalog_manager.cc:5495] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131). New cstate: current_term: 1 leader_uuid: "44ba16574fc54f1180edcab741b6d4bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } health_report { overall_health: HEALTHY } } }
I20260809 16:30:04.515911 11730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.513210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55162 (local address 127.30.23.131:36445)
0809 16:30:04.513266 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.513269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.513278 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:30:04.513322 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.513323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.513382 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.513468 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:30:04.513609 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.514079 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.515149 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.515333 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.515392 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.515579 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:30:04.515676 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:30:04.515728 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.515804 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.515824 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:30:04.516997 11352 raft_consensus.cc:1261] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Refusing update from remote peer 44ba16574fc54f1180edcab741b6d4bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:04.517012 11109 raft_consensus.cc:1261] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Refusing update from remote peer 44ba16574fc54f1180edcab741b6d4bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:04.517274 11744 consensus_queue.cc:1035] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:04.517475 11868 consensus_queue.cc:1035] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:04.521587 10837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:30:04.521693 10837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:30:04.522432 10837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7: Sending DeleteTablet for 3 replicas of tablet 7400bc6fd9794a3c948cb2313d4814bb
I20260809 16:30:04.522687 11321 tablet_service.cc:1514] Processing DeleteTablet for tablet 7400bc6fd9794a3c948cb2313d4814bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:04 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:45296
I20260809 16:30:04.522701 11066 tablet_service.cc:1514] Processing DeleteTablet for tablet 7400bc6fd9794a3c948cb2313d4814bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:04 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:47362
I20260809 16:30:04.522781 11784 tablet_replica.cc:330] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd: stopping tablet replica
I20260809 16:30:04.522805 11780 tablet_replica.cc:330] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f: stopping tablet replica
I20260809 16:30:04.522845 11784 raft_consensus.cc:2229] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:30:04.522861 11780 raft_consensus.cc:2229] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:30:04.522893 11784 raft_consensus.cc:2258] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:04.522915 11780 raft_consensus.cc:2258] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:04.523098 11579 tablet_service.cc:1514] Processing DeleteTablet for tablet 7400bc6fd9794a3c948cb2313d4814bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:04 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:55160
I20260809 16:30:04.523204 11782 tablet_replica.cc:330] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf: stopping tablet replica
I20260809 16:30:04.523262 11782 raft_consensus.cc:2229] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:30:04.523367 11782 raft_consensus.cc:2258] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:04.523427 10837 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:30:04.523581 10837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:30:04.523677 11780 ts_tablet_manager.cc:1905] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:04.523823 11782 ts_tablet_manager.cc:1905] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:04.525403 11782 ts_tablet_manager.cc:1918] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:30:04.525466 11782 log.cc:1196] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/7400bc6fd9794a3c948cb2313d4814bb
I20260809 16:30:04.525664 11780 ts_tablet_manager.cc:1918] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:30:04.525699 11782 ts_tablet_manager.cc:1939] T 7400bc6fd9794a3c948cb2313d4814bb P 44ba16574fc54f1180edcab741b6d4bf: Deleting consensus metadata
I20260809 16:30:04.525745 11780 log.cc:1196] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-0/wal/wals/7400bc6fd9794a3c948cb2313d4814bb
I20260809 16:30:04.525974 11780 ts_tablet_manager.cc:1939] T 7400bc6fd9794a3c948cb2313d4814bb P eba6ed2ba52344f3b20cba56e29f8f4f: Deleting consensus metadata
I20260809 16:30:04.526302 10810 catalog_manager.cc:4841] TS 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131:36445): tablet 7400bc6fd9794a3c948cb2313d4814bb (table test-table [id=297f4d8e2a5742a68ddf84fb093f54ae]) successfully deleted
I20260809 16:30:04.526595 10813 catalog_manager.cc:4841] TS eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129:45575): tablet 7400bc6fd9794a3c948cb2313d4814bb (table test-table [id=297f4d8e2a5742a68ddf84fb093f54ae]) successfully deleted
I20260809 16:30:04.526618 11321 tablet_service.cc:1467] Processing CreateTablet for tablet 4a799ad1739c4456a8c1986b593a780a (DEFAULT_TABLE table=test-table [id=c675fd2fa1814a00a39817ecf8bdb7fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:04.526769 11321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a799ad1739c4456a8c1986b593a780a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.526798 11066 tablet_service.cc:1467] Processing CreateTablet for tablet 4a799ad1739c4456a8c1986b593a780a (DEFAULT_TABLE table=test-table [id=c675fd2fa1814a00a39817ecf8bdb7fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:04.526906 11066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a799ad1739c4456a8c1986b593a780a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.528136 11741 tablet_bootstrap.cc:492] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd: Bootstrap starting.
I20260809 16:30:04.528183 11579 tablet_service.cc:1467] Processing CreateTablet for tablet 4a799ad1739c4456a8c1986b593a780a (DEFAULT_TABLE table=test-table [id=c675fd2fa1814a00a39817ecf8bdb7fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:04.528285 11579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a799ad1739c4456a8c1986b593a780a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:04.528471 11741 tablet_bootstrap.cc:654] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.528806 11735 tablet_bootstrap.cc:492] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f: Bootstrap starting.
I20260809 16:30:04.529186 11735 tablet_bootstrap.cc:654] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.529395 11734 tablet_bootstrap.cc:492] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf: Bootstrap starting.
I20260809 16:30:04.529749 11741 tablet_bootstrap.cc:492] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd: No bootstrap required, opened a new log
I20260809 16:30:04.529798 11741 ts_tablet_manager.cc:1397] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:30:04.529860 11734 tablet_bootstrap.cc:654] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:04.529904 11741 raft_consensus.cc:348] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.529981 11741 raft_consensus.cc:374] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.529999 11741 raft_consensus.cc:729] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b60fe0d3864708862a8c5256a586fd, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.530036 11741 consensus_queue.cc:260] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.530118 11741 ts_tablet_manager.cc:1428] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:04.530893 11734 tablet_bootstrap.cc:492] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf: No bootstrap required, opened a new log
I20260809 16:30:04.530943 11734 ts_tablet_manager.cc:1397] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:30:04.531075 11734 raft_consensus.cc:348] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.531132 11734 raft_consensus.cc:374] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.531150 11734 raft_consensus.cc:729] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ba16574fc54f1180edcab741b6d4bf, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.531189 11734 consensus_queue.cc:260] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.531303 11734 ts_tablet_manager.cc:1428] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:04.533651 11784 ts_tablet_manager.cc:1905] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:04.535493 11784 ts_tablet_manager.cc:1918] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:30:04.535566 11784 log.cc:1196] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/7400bc6fd9794a3c948cb2313d4814bb
I20260809 16:30:04.535801 11784 ts_tablet_manager.cc:1939] T 7400bc6fd9794a3c948cb2313d4814bb P a9b60fe0d3864708862a8c5256a586fd: Deleting consensus metadata
I20260809 16:30:04.535854 11735 tablet_bootstrap.cc:492] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f: No bootstrap required, opened a new log
I20260809 16:30:04.535912 11735 ts_tablet_manager.cc:1397] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260809 16:30:04.536062 11735 raft_consensus.cc:348] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.536127 11735 raft_consensus.cc:374] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:04.536145 11735 raft_consensus.cc:729] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba6ed2ba52344f3b20cba56e29f8f4f, State: Initialized, Role: FOLLOWER
I20260809 16:30:04.536162 10811 catalog_manager.cc:4841] TS a9b60fe0d3864708862a8c5256a586fd (127.30.23.130:34291): tablet 7400bc6fd9794a3c948cb2313d4814bb (table test-table [id=297f4d8e2a5742a68ddf84fb093f54ae]) successfully deleted
I20260809 16:30:04.536193 11735 consensus_queue.cc:260] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.536289 11735 ts_tablet_manager.cc:1428] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:30:04.685163 11749 raft_consensus.cc:482] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:30:04.685285 11749 raft_consensus.cc:504] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.685629 11749 leader_election.cc:288] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129:45575), 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131:36445)
I20260809 16:30:04.685786 11109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a799ad1739c4456a8c1986b593a780a" candidate_uuid: "a9b60fe0d3864708862a8c5256a586fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" is_pre_election: true
I20260809 16:30:04.685762 11616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a799ad1739c4456a8c1986b593a780a" candidate_uuid: "a9b60fe0d3864708862a8c5256a586fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ba16574fc54f1180edcab741b6d4bf" is_pre_election: true
I20260809 16:30:04.685879 11109 raft_consensus.cc:2454] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b60fe0d3864708862a8c5256a586fd in term 0.
I20260809 16:30:04.685925 11616 raft_consensus.cc:2454] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b60fe0d3864708862a8c5256a586fd in term 0.
I20260809 16:30:04.686116 11233 leader_election.cc:302] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ba16574fc54f1180edcab741b6d4bf, a9b60fe0d3864708862a8c5256a586fd; no voters:
I20260809 16:30:04.686287 11749 raft_consensus.cc:2781] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:30:04.686347 11749 raft_consensus.cc:482] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:30:04.686379 11749 raft_consensus.cc:3037] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.687163 11749 raft_consensus.cc:504] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.687361 11749 leader_election.cc:288] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [CANDIDATE]: Term 1 election: Requested vote from peers eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129:45575), 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131:36445)
I20260809 16:30:04.687527 11616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a799ad1739c4456a8c1986b593a780a" candidate_uuid: "a9b60fe0d3864708862a8c5256a586fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ba16574fc54f1180edcab741b6d4bf"
I20260809 16:30:04.687549 11109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a799ad1739c4456a8c1986b593a780a" candidate_uuid: "a9b60fe0d3864708862a8c5256a586fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f"
I20260809 16:30:04.687629 11616 raft_consensus.cc:3037] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.687628 11109 raft_consensus.cc:3037] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:04.688270 11616 raft_consensus.cc:2454] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b60fe0d3864708862a8c5256a586fd in term 1.
I20260809 16:30:04.688396 11109 raft_consensus.cc:2454] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b60fe0d3864708862a8c5256a586fd in term 1.
I20260809 16:30:04.688463 11233 leader_election.cc:302] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ba16574fc54f1180edcab741b6d4bf, a9b60fe0d3864708862a8c5256a586fd; no voters:
I20260809 16:30:04.688562 11749 raft_consensus.cc:2781] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:30:04.688638 11749 raft_consensus.cc:686] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [term 1 LEADER]: Becoming Leader. State: Replica: a9b60fe0d3864708862a8c5256a586fd, State: Running, Role: LEADER
I20260809 16:30:04.688709 11749 consensus_queue.cc:237] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } }
I20260809 16:30:04.689432 10837 catalog_manager.cc:5495] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd reported cstate change: term changed from 0 to 1, leader changed from <none> to a9b60fe0d3864708862a8c5256a586fd (127.30.23.130). New cstate: current_term: 1 leader_uuid: "a9b60fe0d3864708862a8c5256a586fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9b60fe0d3864708862a8c5256a586fd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 } health_report { overall_health: UNKNOWN } } }
I20260809 16:30:04.714457 11733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.712028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45318 (local address 127.30.23.130:34291)
0809 16:30:04.712186 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:04.712188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:04.712196 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:30:04.712217 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:04.712218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:04.712228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:04.712285 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:30:04.712457 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.712941 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.713852 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:04.714039 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:04.714088 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:04.714236 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:30:04.714264 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:30:04.714304 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:04.714382 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 16:30:04.714399 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260809 16:30:04.716792 11109 raft_consensus.cc:1261] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Refusing update from remote peer a9b60fe0d3864708862a8c5256a586fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:04.716779 11616 raft_consensus.cc:1261] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Refusing update from remote peer a9b60fe0d3864708862a8c5256a586fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:04.716995 11749 consensus_queue.cc:1035] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "44ba16574fc54f1180edcab741b6d4bf" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 36445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:04.717087 11746 consensus_queue.cc:1035] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "eba6ed2ba52344f3b20cba56e29f8f4f" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:04.728628 10837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:30:04.728797 10837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:30:04.729609 10837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acfd262f61ed440db22479a7342d7ac7: Sending DeleteTablet for 3 replicas of tablet 4a799ad1739c4456a8c1986b593a780a
I20260809 16:30:04.729882 11066 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a799ad1739c4456a8c1986b593a780a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:04 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:47362
I20260809 16:30:04.729912 11579 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a799ad1739c4456a8c1986b593a780a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:04 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:55160
I20260809 16:30:04.729900 11321 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a799ad1739c4456a8c1986b593a780a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:04 UTC) from {username='oryx', principal='oryx/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:45296
I20260809 16:30:04.730001 11780 tablet_replica.cc:330] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f: stopping tablet replica
I20260809 16:30:04.730019 11784 tablet_replica.cc:330] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd: stopping tablet replica
I20260809 16:30:04.730057 11780 raft_consensus.cc:2229] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:30:04.730078 11784 raft_consensus.cc:2229] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:30:04.730096 11780 raft_consensus.cc:2258] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:04.730146 11784 raft_consensus.cc:2258] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:04.730163 11782 tablet_replica.cc:330] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf: stopping tablet replica
I20260809 16:30:04.730213 11782 raft_consensus.cc:2229] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:30:04.730257 11782 raft_consensus.cc:2258] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:04.730460 11784 ts_tablet_manager.cc:1905] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:04.730602 11782 ts_tablet_manager.cc:1905] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:04.730794 11780 ts_tablet_manager.cc:1905] T 4a799ad1739c4456a8c1986b593a780a P eba6ed2ba52344f3b20cba56e29f8f4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:04.732096 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 10953
I20260809 16:30:04.732273 11784 ts_tablet_manager.cc:1918] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:30:04.732337 11784 log.cc:1196] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/4a799ad1739c4456a8c1986b593a780a
I20260809 16:30:04.732586 11784 ts_tablet_manager.cc:1939] T 4a799ad1739c4456a8c1986b593a780a P a9b60fe0d3864708862a8c5256a586fd: Deleting consensus metadata
I20260809 16:30:04.735922 10811 catalog_manager.cc:4841] TS a9b60fe0d3864708862a8c5256a586fd (127.30.23.130:34291): tablet 4a799ad1739c4456a8c1986b593a780a (table test-table [id=c675fd2fa1814a00a39817ecf8bdb7fa]) successfully deleted
I20260809 16:30:04.738897 11782 ts_tablet_manager.cc:1918] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:30:04.738958 11782 log.cc:1196] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/4a799ad1739c4456a8c1986b593a780a
I20260809 16:30:04.739145 11782 ts_tablet_manager.cc:1939] T 4a799ad1739c4456a8c1986b593a780a P 44ba16574fc54f1180edcab741b6d4bf: Deleting consensus metadata
I20260809 16:30:04.739514 10810 catalog_manager.cc:4841] TS 44ba16574fc54f1180edcab741b6d4bf (127.30.23.131:36445): tablet 4a799ad1739c4456a8c1986b593a780a (table test-table [id=c675fd2fa1814a00a39817ecf8bdb7fa]) successfully deleted
W20260809 16:30:04.746134 10813 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:45575 (error 108)
I20260809 16:30:04.746426 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 11208
I20260809 16:30:04.751044 11895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:04.746568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45575 (local address 127.0.0.1:47388)
0809 16:30:04.750904 (+ 4336us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:04.750980 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:45575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4310,"thread_start_us":41,"threads_started":1}
W20260809 16:30:04.751210 10813 catalog_manager.cc:4584] TS eba6ed2ba52344f3b20cba56e29f8f4f (127.30.23.129:45575): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a799ad1739c4456a8c1986b593a780a: Network error: Client connection negotiation failed: client connection to 127.30.23.129:45575: connect: Connection refused (error 111)
I20260809 16:30:04.764142 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 11466
I20260809 16:30:04.783807 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 10781
2026-08-09T16:30:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2964 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:30:04 dist-test-slave-j2vl krb5kdc[11902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:30:04 dist-test-slave-j2vl krb5kdc[11902](info): set up 2 sockets
Aug 09 16:30:04 dist-test-slave-j2vl krb5kdc[11902](info): commencing operation
krb5kdc: starting...
W20260809 16:30:06.098917 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:30:06 dist-test-slave-j2vl krb5kdc[11902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:30:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:30:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:30:06.234475 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:45513
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:33561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:45513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:06.318840 11927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:06.319065 11927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:06.319088 11927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:06.320241 11927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:30:06.320286 11927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:06.320300 11927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:06.320313 11927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:30:06.320325 11927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:30:06.321900 11927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:33561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:45513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:45513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:06.322180 11927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:06.322444 11927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:06.327865 11943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:06.327848 11938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:06.327853 11939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:06.327989 11927 server_base.cc:1034] running on GCE node
I20260809 16:30:06.328299 11927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:06.328611 11927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:06.329779 11927 hybrid_clock.cc:648] HybridClock initialized: now 1786293006329762 us; error 46 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T16:30:06Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1538 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:30:06 dist-test-slave-j2vl krb5kdc[11949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:30:06 dist-test-slave-j2vl krb5kdc[11949](info): set up 2 sockets
Aug 09 16:30:06 dist-test-slave-j2vl krb5kdc[11949](info): commencing operation
krb5kdc: starting...
W20260809 16:30:07.606848 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:30:07 dist-test-slave-j2vl krb5kdc[11949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:30:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:30:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:30:07.741088 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:40095
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:40095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:07.825464 11978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:07.825675 11978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:07.825698 11978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:07.826895 11978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:30:07.826942 11978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:07.826957 11978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:07.827000 11978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:30:07.827029 11978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:30:07.828509 11978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:40095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:40095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:07.828781 11978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:07.829015 11978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:07.833914 11991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:07.833915 11994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:07.833973 11978 server_base.cc:1034] running on GCE node
W20260809 16:30:07.833945 11988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:07.834338 11978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:07.834676 11978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:07.835856 11978 hybrid_clock.cc:648] HybridClock initialized: now 1786293007835830 us; error 38 us; skew 500 ppm
Aug 09 16:30:07 dist-test-slave-j2vl krb5kdc[11949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293007, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:07.843847 11978 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:30:07.844394 11978 webserver.cc:460] Webserver started at http://127.30.23.190:40019/ using document root <none> and password file <none>
I20260809 16:30:07.844651 11978 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:07.844723 11978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:07.844883 11978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:07.846079 11978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "9dfcadb60f7d4d8885cb4a84aedd6010"
format_stamp: "Formatted at 2026-08-09 16:30:07 on dist-test-slave-j2vl"
server_key: "f7eadf471e90f7a2baf1339a64e787bf"
server_key_iv: "64b2a14682a9c53d35de2a3c60efdee3"
server_key_version: "encryptionkey@0"
I20260809 16:30:07.846451 11978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "9dfcadb60f7d4d8885cb4a84aedd6010"
format_stamp: "Formatted at 2026-08-09 16:30:07 on dist-test-slave-j2vl"
server_key: "f7eadf471e90f7a2baf1339a64e787bf"
server_key_iv: "64b2a14682a9c53d35de2a3c60efdee3"
server_key_version: "encryptionkey@0"
I20260809 16:30:07.847865 11978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:30:07.848641 12010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:07.848858 11978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:07.848970 11978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "9dfcadb60f7d4d8885cb4a84aedd6010"
format_stamp: "Formatted at 2026-08-09 16:30:07 on dist-test-slave-j2vl"
server_key: "f7eadf471e90f7a2baf1339a64e787bf"
server_key_iv: "64b2a14682a9c53d35de2a3c60efdee3"
server_key_version: "encryptionkey@0"
I20260809 16:30:07.849051 11978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:07.875424 11978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:07.878228 11978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:07.878420 11978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:07.882915 11978 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:40095
I20260809 16:30:07.882977 12101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:40095 every 8 connection(s)
I20260809 16:30:07.883378 11978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:30:07.884300 12102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:07.886724 12102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010: Bootstrap starting.
I20260809 16:30:07.887112 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 11978
I20260809 16:30:07.887282 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:30:07.887462 12102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:07.887527 30814 external_mini_cluster.cc:1428] Setting key ddc0f56d34badd8890db19b04ecdad95
I20260809 16:30:07.887775 12102 log.cc:824] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010: Log is configured to *not* fsync() on all Append() calls
I20260809 16:30:07.888652 12102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010: No bootstrap required, opened a new log
I20260809 16:30:07.890944 12102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40095 } }
I20260809 16:30:07.891149 12102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:07.891189 12102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dfcadb60f7d4d8885cb4a84aedd6010, State: Initialized, Role: FOLLOWER
I20260809 16:30:07.891315 12102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40095 } }
I20260809 16:30:07.891398 12102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:30:07.891429 12102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:30:07.891476 12102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:07.892170 12102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40095 } }
I20260809 16:30:07.892261 12102 leader_election.cc:302] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9dfcadb60f7d4d8885cb4a84aedd6010; no voters:
I20260809 16:30:07.892462 12102 leader_election.cc:288] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:30:07.892520 12109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:30:07.892745 12102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:30:07.892765 12109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 1 LEADER]: Becoming Leader. State: Replica: 9dfcadb60f7d4d8885cb4a84aedd6010, State: Running, Role: LEADER
I20260809 16:30:07.892906 12109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40095 } }
I20260809 16:30:07.893329 12111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dfcadb60f7d4d8885cb4a84aedd6010. Latest consensus state: current_term: 1 leader_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40095 } } }
I20260809 16:30:07.893366 12110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40095 } } }
I20260809 16:30:07.893544 12111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [sys.catalog]: This master's current role is: LEADER
I20260809 16:30:07.893657 12110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [sys.catalog]: This master's current role is: LEADER
I20260809 16:30:07.893957 12117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:30:07.894554 12117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:30:07 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:30:07.899511 12108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:07.888110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45202 (local address 127.30.23.190:40095)
0809 16:30:07.888303 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:07.888314 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:07.888358 (+ 44us) server_negotiation.cc:407] Connection header received
0809 16:30:07.888915 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:07.888924 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:07.888978 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:07.889115 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:07.889995 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:07.890891 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:07.891961 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:07.892605 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:07.893091 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:07.895937 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:07.895941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:07.895942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:07.895952 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:07.897823 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:07.898251 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:07.898253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:07.898254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:07.898340 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:07.898582 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:07.898583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:07.898584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:07.898893 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:07.899113 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:07.899184 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 16:30:07.899326 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260809 16:30:07.899669 12117 catalog_manager.cc:1347] Generated new cluster ID: 6066fdd95c644a088996bc08a350d7d5
I20260809 16:30:07.899721 12117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:30:07.923722 12117 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:30:07.924508 12117 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:30:07.938612 12117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010: Generated new TSK 0
I20260809 16:30:07.938835 12117 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:30:07.980512 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:40095
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:08.066463 12142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:08.066685 12142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:08.066712 12142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:08.067956 12142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:08.068004 12142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:08.068032 12142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:30:08.069564 12142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:40095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:08.069820 12142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:08.070077 12142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:08.075348 12157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:08.075441 12151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:08.075404 12152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:08.075681 12142 server_base.cc:1034] running on GCE node
I20260809 16:30:08.075917 12142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:08.076184 12142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:08.077423 12142 hybrid_clock.cc:648] HybridClock initialized: now 1786293008077396 us; error 43 us; skew 500 ppm
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:08.080276 12142 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:30:08.080757 12142 webserver.cc:460] Webserver started at http://127.30.23.129:34145/ using document root <none> and password file <none>
I20260809 16:30:08.081012 12142 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:08.081085 12142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:08.081243 12142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:08.082396 12142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "64a055351a9b4c63859a8c9f18b50820"
format_stamp: "Formatted at 2026-08-09 16:30:08 on dist-test-slave-j2vl"
server_key: "23f9c2a5814629966582405200620aa1"
server_key_iv: "0cc82942c40ee73cc6638d03e630d28d"
server_key_version: "encryptionkey@0"
I20260809 16:30:08.082780 12142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "64a055351a9b4c63859a8c9f18b50820"
format_stamp: "Formatted at 2026-08-09 16:30:08 on dist-test-slave-j2vl"
server_key: "23f9c2a5814629966582405200620aa1"
server_key_iv: "0cc82942c40ee73cc6638d03e630d28d"
server_key_version: "encryptionkey@0"
I20260809 16:30:08.084383 12142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:30:08.085230 12171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.085480 12142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:08.085593 12142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "64a055351a9b4c63859a8c9f18b50820"
format_stamp: "Formatted at 2026-08-09 16:30:08 on dist-test-slave-j2vl"
server_key: "23f9c2a5814629966582405200620aa1"
server_key_iv: "0cc82942c40ee73cc6638d03e630d28d"
server_key_version: "encryptionkey@0"
I20260809 16:30:08.085680 12142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:08.108435 12142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:08.111104 12142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:08.111300 12142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:08.111937 12142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:08.112342 12142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:08.112401 12142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.112452 12142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:08.112497 12142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.118634 12142 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:44089
I20260809 16:30:08.118726 12367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:44089 every 8 connection(s)
I20260809 16:30:08.119063 12142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:30:08.126775 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 12142
I20260809 16:30:08.126893 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:30:08.127148 30814 external_mini_cluster.cc:1428] Setting key 09d3e88fab6c03bc4fa86a782a48208b
I20260809 16:30:08.129506 12108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:08.119826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:52369 (local address 127.30.23.190:40095)
0809 16:30:08.119879 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:08.119881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:08.120084 (+ 203us) server_negotiation.cc:407] Connection header received
0809 16:30:08.120610 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:08.120612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:08.120625 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:08.120693 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:08.121785 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.122306 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.123295 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.123509 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.123574 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:08.125979 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:08.125981 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:08.125982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:08.125990 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:08.127755 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:08.128263 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:08.128265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:08.128265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:08.128361 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:08.128697 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:08.128697 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:08.128698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:08.128841 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:08.128911 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:08.129284 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 16:30:08.129425 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:30:08.129757 12372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:08.119788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:52369)
0809 16:30:08.119965 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:08.120009 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:08.120491 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:08.120772 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:08.120776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:08.121460 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:08.121667 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:08.121672 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.122394 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.122395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:08.123195 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:08.123197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.123611 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.123612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:08.123724 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:08.124344 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:08.124352 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:08.125862 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:08.127837 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:08.127846 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:08.127849 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:08.128163 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:08.128464 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:08.128465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:08.128466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:08.128619 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:08.128915 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:08.128922 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:08.129122 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 16:30:08.129244 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 16:30:08.129466 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260809 16:30:08.130224 12369 heartbeater.cc:347] Connected to a master server at 127.30.23.190:40095
I20260809 16:30:08.130326 12369 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:08.130541 12369 heartbeater.cc:511] Master 127.30.23.190:40095 requested a full tablet report, sending...
I20260809 16:30:08.131067 12035 ts_manager.cc:194] Registered new tserver with Master: 64a055351a9b4c63859a8c9f18b50820 (127.30.23.129:44089)
I20260809 16:30:08.131703 12035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:52369
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:30:08.156919 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:40095
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:08.239674 12375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:08.239876 12375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:08.239900 12375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:08.241001 12375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:08.241037 12375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:08.241062 12375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:30:08.242529 12375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:40095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:08.242730 12375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:08.242914 12375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:08.248065 12384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:08.248049 12389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:08.248029 12386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:08.248270 12375 server_base.cc:1034] running on GCE node
I20260809 16:30:08.248512 12375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:08.248752 12375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:08.249940 12375 hybrid_clock.cc:648] HybridClock initialized: now 1786293008249918 us; error 29 us; skew 500 ppm
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:08.252728 12375 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:30:08.253198 12375 webserver.cc:460] Webserver started at http://127.30.23.130:33437/ using document root <none> and password file <none>
I20260809 16:30:08.253505 12375 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:08.253575 12375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:08.253741 12375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:08.254809 12375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "f8836ee60c544ac798d05fb824f1976c"
format_stamp: "Formatted at 2026-08-09 16:30:08 on dist-test-slave-j2vl"
server_key: "2f42b70e17825e60086f8ae23fce8cdb"
server_key_iv: "36bc7413ad56b9d510ec5e05ccf19daa"
server_key_version: "encryptionkey@0"
I20260809 16:30:08.255182 12375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "f8836ee60c544ac798d05fb824f1976c"
format_stamp: "Formatted at 2026-08-09 16:30:08 on dist-test-slave-j2vl"
server_key: "2f42b70e17825e60086f8ae23fce8cdb"
server_key_iv: "36bc7413ad56b9d510ec5e05ccf19daa"
server_key_version: "encryptionkey@0"
I20260809 16:30:08.256633 12375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:30:08.257433 12405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.257615 12375 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:08.257718 12375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "f8836ee60c544ac798d05fb824f1976c"
format_stamp: "Formatted at 2026-08-09 16:30:08 on dist-test-slave-j2vl"
server_key: "2f42b70e17825e60086f8ae23fce8cdb"
server_key_iv: "36bc7413ad56b9d510ec5e05ccf19daa"
server_key_version: "encryptionkey@0"
I20260809 16:30:08.257807 12375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:08.269678 12375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:08.272047 12375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:08.272217 12375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:08.272460 12375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:08.272782 12375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:08.272845 12375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.272888 12375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:08.272915 12375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.278895 12375 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:43659
I20260809 16:30:08.278946 12598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:43659 every 8 connection(s)
I20260809 16:30:08.279315 12375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:30:08.282591 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 12375
I20260809 16:30:08.282706 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:30:08.282922 30814 external_mini_cluster.cc:1428] Setting key 05689d243da8744a2245a0c815e4a6f1
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:30:08.290722 12108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:08.280208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:44447 (local address 127.30.23.190:40095)
0809 16:30:08.280302 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:08.280304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:08.280483 (+ 179us) server_negotiation.cc:407] Connection header received
0809 16:30:08.280951 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:08.280952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:08.280963 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:08.281018 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:08.282076 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.282771 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.284143 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.284475 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.284532 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:08.287369 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:08.287371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:08.287371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:08.287378 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:08.288952 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:08.289526 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:08.289527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:08.289527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:08.289584 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:08.289971 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:08.289973 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:08.289973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:08.290106 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:08.290173 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:08.290575 (+ 402us) server_negotiation.cc:299] Negotiation successful
0809 16:30:08.290649 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 16:30:08.290973 12602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:08.280154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:44447)
0809 16:30:08.280347 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:08.280385 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:08.280860 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:08.281095 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:08.281099 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:08.281772 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:08.281972 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:08.281976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.282864 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.282865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:08.284021 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:08.284024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.284554 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.284555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:08.284707 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:08.285235 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:08.285245 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:08.287240 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:08.289043 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:08.289051 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:08.289053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:08.289448 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:08.289695 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:08.289696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:08.289696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:08.289851 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:08.290186 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:08.290194 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:08.290378 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 16:30:08.290514 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 16:30:08.290708 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":121,"threads_started":1}
I20260809 16:30:08.291507 12600 heartbeater.cc:347] Connected to a master server at 127.30.23.190:40095
I20260809 16:30:08.291628 12600 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:08.291847 12600 heartbeater.cc:511] Master 127.30.23.190:40095 requested a full tablet report, sending...
I20260809 16:30:08.292309 12035 ts_manager.cc:194] Registered new tserver with Master: f8836ee60c544ac798d05fb824f1976c (127.30.23.130:43659)
I20260809 16:30:08.292698 12035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:44447
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:30:08.313752 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:40095
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:08.397606 12606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:08.397806 12606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:08.397830 12606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:08.398967 12606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:08.399009 12606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:08.399035 12606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:30:08.400446 12606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:40095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:08.400691 12606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:08.400928 12606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:08.406229 12616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:08.406229 12611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:08.406255 12612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:08.406670 12606 server_base.cc:1034] running on GCE node
I20260809 16:30:08.406875 12606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:08.407140 12606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:08.408291 12606 hybrid_clock.cc:648] HybridClock initialized: now 1786293008408268 us; error 42 us; skew 500 ppm
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:08.411047 12606 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:30:08.411542 12606 webserver.cc:460] Webserver started at http://127.30.23.131:35917/ using document root <none> and password file <none>
I20260809 16:30:08.411793 12606 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:08.411860 12606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:08.412006 12606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:08.413050 12606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "cf7bc1abdb5343ca9a850703a84ce966"
format_stamp: "Formatted at 2026-08-09 16:30:08 on dist-test-slave-j2vl"
server_key: "1dc09b492fee1e4c26b7f3ff940b2acb"
server_key_iv: "274efecdf7c22a84f28185ee62e9aaca"
server_key_version: "encryptionkey@0"
I20260809 16:30:08.413480 12606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "cf7bc1abdb5343ca9a850703a84ce966"
format_stamp: "Formatted at 2026-08-09 16:30:08 on dist-test-slave-j2vl"
server_key: "1dc09b492fee1e4c26b7f3ff940b2acb"
server_key_iv: "274efecdf7c22a84f28185ee62e9aaca"
server_key_version: "encryptionkey@0"
I20260809 16:30:08.414856 12606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:08.415650 12627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.415827 12606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:30:08.415931 12606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "cf7bc1abdb5343ca9a850703a84ce966"
format_stamp: "Formatted at 2026-08-09 16:30:08 on dist-test-slave-j2vl"
server_key: "1dc09b492fee1e4c26b7f3ff940b2acb"
server_key_iv: "274efecdf7c22a84f28185ee62e9aaca"
server_key_version: "encryptionkey@0"
I20260809 16:30:08.416016 12606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:08.434185 12606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:08.436437 12606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:08.436615 12606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:08.436897 12606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:08.437261 12606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:08.437320 12606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.437387 12606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:08.437436 12606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.443967 12606 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:43561
I20260809 16:30:08.444018 12805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:43561 every 8 connection(s)
I20260809 16:30:08.444376 12606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:30:08.449687 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 12606
I20260809 16:30:08.449852 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:30:08.450066 30814 external_mini_cluster.cc:1428] Setting key 37eab16305c434660c9dd9d5be2100e1
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:30:08.454910 12108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:08.445212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:55665 (local address 127.30.23.190:40095)
0809 16:30:08.445259 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:08.445262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:08.445580 (+ 318us) server_negotiation.cc:407] Connection header received
0809 16:30:08.446119 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:08.446121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:08.446135 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:08.446203 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:08.447194 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.447714 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.448730 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.448956 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.449049 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:08.451934 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:08.451936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:08.451936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:08.451945 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:08.453468 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:08.453884 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:08.453886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:08.453887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:08.453955 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:08.454219 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:08.454220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:08.454220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:08.454344 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:08.454424 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:08.454728 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 16:30:08.454818 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:30:08.455122 12809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:08.445204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:55665)
0809 16:30:08.445427 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:08.445491 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:08.446011 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:08.446274 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:08.446278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:08.446886 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:08.447089 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:08.447093 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.447824 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.447825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:08.448632 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:08.448635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.449062 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.449063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:08.449192 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:08.450029 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:08.450038 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:08.451835 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:08.453550 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:08.453558 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:08.453560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:08.453791 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:08.454026 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:08.454027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:08.454028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:08.454142 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:08.454414 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:08.454421 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:08.454567 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 16:30:08.454675 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 16:30:08.454857 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":91,"threads_started":1}
I20260809 16:30:08.455591 12807 heartbeater.cc:347] Connected to a master server at 127.30.23.190:40095
I20260809 16:30:08.455718 12807 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:08.455914 12807 heartbeater.cc:511] Master 127.30.23.190:40095 requested a full tablet report, sending...
I20260809 16:30:08.456321 12035 ts_manager.cc:194] Registered new tserver with Master: cf7bc1abdb5343ca9a850703a84ce966 (127.30.23.131:43561)
I20260809 16:30:08.456734 12035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:55665
I20260809 16:30:08.462416 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab.
Entry for principal oryx/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab.
WARNING: no policy specified for oryx/127.30.23.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab.
Entry for principal oryx/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab.
WARNING: no policy specified for oryx/127.30.23.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab.
Entry for principal oryx/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab.
I20260809 16:30:08.552910 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 12142
I20260809 16:30:08.564416 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 12375
I20260809 16:30:08.575001 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 12606
I20260809 16:30:08.587184 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 11978
I20260809 16:30:08.597601 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:40095
--webserver_interface=127.30.23.190
--webserver_port=40019
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:40095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:08.681497 12820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:08.681753 12820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:08.681803 12820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:08.683003 12820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:30:08.683074 12820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:08.683101 12820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:08.683131 12820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:30:08.683156 12820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:30:08.684677 12820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:40095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:40095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=40019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:08.684979 12820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:08.685216 12820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:08.690182 12830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:08.690188 12828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:08.690188 12825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:08.690537 12820 server_base.cc:1034] running on GCE node
I20260809 16:30:08.690810 12820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:08.691119 12820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:08.692291 12820 hybrid_clock.cc:648] HybridClock initialized: now 1786293008692276 us; error 34 us; skew 500 ppm
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:08.695034 12820 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:30:08.695495 12820 webserver.cc:460] Webserver started at http://127.30.23.190:40019/ using document root <none> and password file <none>
I20260809 16:30:08.695744 12820 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:08.695832 12820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:08.697227 12820 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:08.698004 12842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.698184 12820 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:30:08.698283 12820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "9dfcadb60f7d4d8885cb4a84aedd6010"
format_stamp: "Formatted at 2026-08-09 16:30:07 on dist-test-slave-j2vl"
server_key: "f7eadf471e90f7a2baf1339a64e787bf"
server_key_iv: "64b2a14682a9c53d35de2a3c60efdee3"
server_key_version: "encryptionkey@0"
I20260809 16:30:08.698655 12820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:08.708722 12820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:08.710829 12820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:08.711009 12820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:08.715624 12820 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:40095
I20260809 16:30:08.715657 12943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:40095 every 8 connection(s)
I20260809 16:30:08.716090 12820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:30:08.717662 12944 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:30:08.718242 12944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010: Bootstrap starting.
I20260809 16:30:08.722695 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 12820
I20260809 16:30:08.723085 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:44089
--local_ip_for_outbound_sockets=127.30.23.129
--tserver_master_addrs=127.30.23.190:40095
--webserver_port=34145
--webserver_interface=127.30.23.129
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.23.129
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:30:08.727566 12944 log.cc:824] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010: Log is configured to *not* fsync() on all Append() calls
I20260809 16:30:08.729310 12944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:30:08.729663 12944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010: Bootstrap complete.
I20260809 16:30:08.732009 12944 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40095 } }
I20260809 16:30:08.732272 12944 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dfcadb60f7d4d8885cb4a84aedd6010, State: Initialized, Role: FOLLOWER
I20260809 16:30:08.732487 12944 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40095 } }
I20260809 16:30:08.732590 12944 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:30:08.732647 12944 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:30:08.732695 12944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:30:08.733814 12944 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40095 } }
I20260809 16:30:08.734108 12944 leader_election.cc:302] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9dfcadb60f7d4d8885cb4a84aedd6010; no voters:
I20260809 16:30:08.734359 12944 leader_election.cc:288] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:30:08.734408 12949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:30:08.734588 12949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [term 2 LEADER]: Becoming Leader. State: Replica: 9dfcadb60f7d4d8885cb4a84aedd6010, State: Running, Role: LEADER
I20260809 16:30:08.734719 12944 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:30:08.734769 12949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40095 } }
I20260809 16:30:08.735180 12950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40095 } } }
I20260809 16:30:08.735277 12950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [sys.catalog]: This master's current role is: LEADER
I20260809 16:30:08.735541 12953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dfcadb60f7d4d8885cb4a84aedd6010. Latest consensus state: current_term: 2 leader_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dfcadb60f7d4d8885cb4a84aedd6010" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40095 } } }
I20260809 16:30:08.735663 12953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010 [sys.catalog]: This master's current role is: LEADER
I20260809 16:30:08.735728 12963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:30:08.736279 12963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:30:08.736490 12963 catalog_manager.cc:1259] Loaded cluster ID: 6066fdd95c644a088996bc08a350d7d5
I20260809 16:30:08.736680 12963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:30:08.738123 12963 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:30:08.738323 12963 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9dfcadb60f7d4d8885cb4a84aedd6010: Loaded TSK: 0
I20260809 16:30:08.738574 12963 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:30:08.814755 12947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:08.815013 12947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:08.815057 12947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:08.816247 12947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:08.816318 12947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:08.816365 12947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:30:08.817939 12947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.129.keytab
--principal=oryx/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:44089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=34145
--tserver_master_addrs=127.30.23.190:40095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:08.818225 12947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:08.818477 12947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:08.825177 12990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:08.825294 12987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:08.825168 12988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:08.825598 12947 server_base.cc:1034] running on GCE node
I20260809 16:30:08.825795 12947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:08.826052 12947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:08.827245 12947 hybrid_clock.cc:648] HybridClock initialized: now 1786293008827222 us; error 30 us; skew 500 ppm
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293008, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:08.830610 12947 init.cc:377] Logged in from keytab as oryx/127.30.23.129@KRBTEST.COM (short username oryx)
I20260809 16:30:08.831187 12947 webserver.cc:460] Webserver started at http://127.30.23.129:34145/ using document root <none> and password file <none>
I20260809 16:30:08.831459 12947 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:08.831557 12947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:08.832980 12947 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:30:08.833657 13003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.833927 12947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:30:08.834031 12947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "64a055351a9b4c63859a8c9f18b50820"
format_stamp: "Formatted at 2026-08-09 16:30:08 on dist-test-slave-j2vl"
server_key: "23f9c2a5814629966582405200620aa1"
server_key_iv: "0cc82942c40ee73cc6638d03e630d28d"
server_key_version: "encryptionkey@0"
I20260809 16:30:08.834664 12947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:08.851039 12947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:08.853552 12947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:08.853717 12947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:08.854036 12947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:08.854425 12947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:08.854477 12947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.854511 12947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:08.854529 12947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.862509 12947 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:44089
I20260809 16:30:08.862594 13205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:44089 every 8 connection(s)
I20260809 16:30:08.863042 12947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:30:08.869658 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 12947
I20260809 16:30:08.870059 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:43659
--local_ip_for_outbound_sockets=127.30.23.130
--tserver_master_addrs=127.30.23.190:40095
--webserver_port=33437
--webserver_interface=127.30.23.130
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.23.130
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:08.875329 13208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:08.863979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:38243)
0809 16:30:08.864177 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:08.864239 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:08.865067 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:08.865678 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:08.865684 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:08.866503 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:08.866784 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:08.866790 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.868312 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.868313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:08.870182 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:08.870186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.871117 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.871119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:08.871289 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:08.872014 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:08.872027 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:08.875107 (+ 3080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
W20260809 16:30:08.875530 13210 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:08.864122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:38243 (local address 127.30.23.190:40095)
0809 16:30:08.864378 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:08.864388 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:08.864432 (+ 44us) server_negotiation.cc:407] Connection header received
0809 16:30:08.865331 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:08.865339 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:08.865442 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:08.865580 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:08.866946 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.868185 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.870320 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.870985 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.871113 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:08.875321 (+ 4208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:38243: BlockingRecv error: recv got EOF from 127.30.23.129:38243 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":131,"threads_started":1}
W20260809 16:30:08.875995 13206 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:40095 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.23.190:40095: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:08.883065 13208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:08.876247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:52679)
0809 16:30:08.876292 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:08.876303 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:08.876340 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:08.876515 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:08.876517 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:08.876686 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:08.876751 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:08.876752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.877785 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.877786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:08.878632 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:08.878634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.879456 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.879457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:08.879528 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:08.880101 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:08.880109 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:08.882989 (+ 2880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 16:30:08.883234 13210 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:08.876322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:52679 (local address 127.30.23.190:40095)
0809 16:30:08.876366 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:08.876369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:08.876378 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:30:08.876411 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:08.876411 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:08.876425 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:08.876501 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:08.876835 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.877700 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.878875 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.879346 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.879437 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:08.883146 (+ 3709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:52679: BlockingRecv error: recv got EOF from 127.30.23.129:52679 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:08.889348 13213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:08.883522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:49339)
0809 16:30:08.883859 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:08.883873 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:08.883966 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:08.884170 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:08.884171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:08.884367 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:08.884444 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:08.884446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.885219 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.885220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:08.886196 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:08.886200 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.886614 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.886616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:08.886691 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:08.887225 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:08.887230 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:08.889231 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":112,"threads_started":1}
W20260809 16:30:08.889529 13206 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:30:08.889585 13210 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:08.883731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:49339 (local address 127.30.23.190:40095)
0809 16:30:08.883779 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:08.883782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:08.883934 (+ 152us) server_negotiation.cc:407] Connection header received
0809 16:30:08.884058 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:08.884060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:08.884075 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:08.884145 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:08.884538 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.885136 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.886288 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:08.886490 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:08.886588 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:08.889325 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:49339: BlockingRecv error: recv got EOF from 127.30.23.129:49339 (error 108)
Metrics: {"server-negotiator.queue_time_us":26,"thread_start_us":84,"threads_started":1}
W20260809 16:30:08.963658 13212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:08.963899 13212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:08.963948 13212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:08.965134 13212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:08.965204 13212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:08.965248 13212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:30:08.966763 13212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.130.keytab
--principal=oryx/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:43659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=33437
--tserver_master_addrs=127.30.23.190:40095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:08.967041 13212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:08.967304 13212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:30:08.973030 13212 server_base.cc:1034] running on GCE node
W20260809 16:30:08.973011 13227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:08.973011 13224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:08.973013 13225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:08.973543 13212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:08.973819 13212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:08.974992 13212 hybrid_clock.cc:648] HybridClock initialized: now 1786293008974978 us; error 52 us; skew 500 ppm
Aug 09 16:30:08 dist-test-slave-j2vl krb5kdc[11949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293008, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:08.977842 13212 init.cc:377] Logged in from keytab as oryx/127.30.23.130@KRBTEST.COM (short username oryx)
I20260809 16:30:08.978314 13212 webserver.cc:460] Webserver started at http://127.30.23.130:33437/ using document root <none> and password file <none>
I20260809 16:30:08.978550 13212 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:08.978636 13212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:08.980145 13212 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:30:08.980904 13239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.981128 13212 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:08.981269 13212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "f8836ee60c544ac798d05fb824f1976c"
format_stamp: "Formatted at 2026-08-09 16:30:08 on dist-test-slave-j2vl"
server_key: "2f42b70e17825e60086f8ae23fce8cdb"
server_key_iv: "36bc7413ad56b9d510ec5e05ccf19daa"
server_key_version: "encryptionkey@0"
I20260809 16:30:08.981685 13212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:08.990844 13212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:08.992820 13212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:08.992987 13212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:08.993227 13212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:08.993604 13212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:08.993660 13212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.993702 13212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:08.993733 13212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:08.999940 13212 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:43659
I20260809 16:30:09.000002 13427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:43659 every 8 connection(s)
I20260809 16:30:09.000376 13212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:09.007728 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 13212
I20260809 16:30:09.008046 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:43561
--local_ip_for_outbound_sockets=127.30.23.131
--tserver_master_addrs=127.30.23.190:40095
--webserver_port=35917
--webserver_interface=127.30.23.131
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.23.131
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:30:09.008347 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:09.001187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:51281)
0809 16:30:09.001409 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:09.001455 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:09.001974 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:09.002308 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:09.002314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:09.002911 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:09.003106 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.003110 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.003828 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.003829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.004632 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.004634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.005017 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.005018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.005165 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.005749 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:09.005758 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:09.008157 (+ 2399us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":71,"threads_started":1}
W20260809 16:30:09.008450 13210 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:09.001196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:51281 (local address 127.30.23.190:40095)
0809 16:30:09.001263 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:09.001265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:09.001541 (+ 276us) server_negotiation.cc:407] Connection header received
0809 16:30:09.002086 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:09.002087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:09.002134 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:09.002210 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:09.003213 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.003721 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.004727 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.004932 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.005008 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.008366 (+ 3358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:51281: BlockingRecv error: recv got EOF from 127.30.23.130:51281 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 16:30:09.008611 13428 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:40095 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.23.190:40095: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:09.013769 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:09.008849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:48687)
0809 16:30:09.008902 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:09.008913 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:09.008957 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:09.009086 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:09.009086 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:09.009199 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:09.009243 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.009244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.010168 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.010170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.010674 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.010676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.011025 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.011026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.011089 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.011671 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:09.011676 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:09.013667 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 16:30:09.013810 13210 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:09.008902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:48687 (local address 127.30.23.190:40095)
0809 16:30:09.008943 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:09.008945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:09.008955 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:30:09.009004 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:09.009005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:09.009015 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:09.009079 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:09.009308 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.010075 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.010758 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.010953 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.011027 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.013766 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:48687: BlockingRecv error: recv got EOF from 127.30.23.130:48687 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:09.019294 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:09.014185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:45585)
0809 16:30:09.014266 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:09.014281 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:09.014335 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:09.014519 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:09.014520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:09.014660 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:09.014691 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.014692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.015424 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.015425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.015961 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.015963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.016348 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.016349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.016533 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.017040 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:09.017045 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:09.019195 (+ 2150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 16:30:09.019444 13210 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:09.014327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:45585 (local address 127.30.23.190:40095)
0809 16:30:09.014388 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:09.014390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:09.014400 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:30:09.014435 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:09.014436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:09.014444 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:09.014514 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:09.014819 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.015338 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.016051 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.016267 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.016351 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.019323 (+ 2972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:45585: BlockingRecv error: recv got EOF from 127.30.23.130:45585 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 16:30:09.019438 13428 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:30:09.095810 13436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:09.096030 13436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:09.096055 13436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:09.097281 13436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:09.097330 13436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:09.097425 13436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:30:09.098935 13436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/krb5kdc/oryx_127.30.23.131.keytab
--principal=oryx/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:43561
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=35917
--tserver_master_addrs=127.30.23.190:40095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:09.099143 13436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:09.099341 13436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:09.105073 13445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:09.105152 13436 server_base.cc:1034] running on GCE node
W20260809 16:30:09.105091 13453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:09.105082 13446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:09.105545 13436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:09.105818 13436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:09.106967 13436 hybrid_clock.cc:648] HybridClock initialized: now 1786293009106955 us; error 30 us; skew 500 ppm
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293009, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:09.109684 13436 init.cc:377] Logged in from keytab as oryx/127.30.23.131@KRBTEST.COM (short username oryx)
I20260809 16:30:09.110194 13436 webserver.cc:460] Webserver started at http://127.30.23.131:35917/ using document root <none> and password file <none>
I20260809 16:30:09.110440 13436 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:09.110534 13436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:09.111968 13436 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:09.112628 13462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:09.112820 13436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:30:09.112928 13436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "cf7bc1abdb5343ca9a850703a84ce966"
format_stamp: "Formatted at 2026-08-09 16:30:08 on dist-test-slave-j2vl"
server_key: "1dc09b492fee1e4c26b7f3ff940b2acb"
server_key_iv: "274efecdf7c22a84f28185ee62e9aaca"
server_key_version: "encryptionkey@0"
I20260809 16:30:09.113282 13436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:09.130702 13436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:09.133281 13436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:09.133505 13436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:09.133767 13436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:09.134107 13436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:09.134164 13436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:09.134209 13436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:09.134258 13436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:09.142714 13436 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:43561
I20260809 16:30:09.142796 13635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:43561 every 8 connection(s)
I20260809 16:30:09.143222 13436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:30:09.144609 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 13436
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 16:30:09.152768 13210 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:09.144105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:59197 (local address 127.30.23.190:40095)
0809 16:30:09.144173 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:09.144175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:09.144393 (+ 218us) server_negotiation.cc:407] Connection header received
0809 16:30:09.145115 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:09.145117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:09.145129 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:09.145171 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:09.146560 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.147369 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.148781 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.149019 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.149067 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.152692 (+ 3625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:59197: BlockingRecv error: recv got EOF from 127.30.23.131:59197 (error 108)
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":36}
I20260809 16:30:09.152778 13641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:09.144091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:59197)
0809 16:30:09.144246 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:09.144290 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:09.144882 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:09.145251 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:09.145257 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:09.146144 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:09.146470 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.146477 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.147499 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.147501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.148707 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.148710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.149108 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.149109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.149263 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.149754 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:09.149765 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:09.152564 (+ 2799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
W20260809 16:30:09.153231 13636 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:40095 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.23.190:40095: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
I20260809 16:30:09.154532 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:09.145486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45216 (local address 127.30.23.190:40095)
0809 16:30:09.145553 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:09.145555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:09.145562 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:30:09.145588 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:09.145588 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:09.145597 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:09.145654 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:09.146490 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.147369 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.148550 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.148783 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.148892 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.150595 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:09.150600 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:09.150601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:09.150615 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:09.152620 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:09.153095 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:09.153099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:09.153100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:09.153241 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:09.153605 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:09.153606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:09.153606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:09.153905 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:09.154077 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:09.154249 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 16:30:09.154384 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":33}
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:09.160257 13641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:09.153545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:48851)
0809 16:30:09.153603 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:09.153616 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:09.153675 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:09.153832 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:09.153833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:09.153988 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:09.154037 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.154038 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.155937 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.155938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.156779 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.156782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.157163 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.157164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.157230 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.157750 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:09.157755 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:09.160162 (+ 2407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 16:30:09.160318 13210 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:09.153603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:48851 (local address 127.30.23.190:40095)
0809 16:30:09.153666 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:09.153668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:09.153678 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:30:09.153736 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:09.153737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:09.153747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:09.153814 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:09.154128 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.154912 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.156868 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.157088 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.157166 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.160253 (+ 3087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:48851: BlockingRecv error: recv got EOF from 127.30.23.131:48851 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:09.165016 13641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:09.160543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:39929)
0809 16:30:09.160597 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:09.160610 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:09.160653 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:09.160826 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:09.160827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:09.160968 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:09.161000 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.161001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.161743 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.161744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.162242 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.162243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.162607 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.162609 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.162663 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.163065 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:09.163070 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:09.164924 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 16:30:09.165069 13210 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:09.160630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:39929 (local address 127.30.23.190:40095)
0809 16:30:09.160707 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:09.160709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:09.160716 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:30:09.160739 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:09.160740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:09.160749 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:09.160821 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:09.161085 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.161654 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.162326 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.162532 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.162610 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.165002 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:39929: BlockingRecv error: recv got EOF from 127.30.23.131:39929 (error 108)
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":59}
W20260809 16:30:09.165167 13636 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:09 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:09.895917 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:09.889990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:54589)
0809 16:30:09.890254 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:09.890267 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:09.890333 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:09.890494 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:09.890495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:09.890679 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:09.890735 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.890736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.891722 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.891723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.892283 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:09.892286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.892864 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.892866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:09.893135 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.893820 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:09.893825 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:09.895852 (+ 2027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":177,"threads_started":1}
W20260809 16:30:09.896061 13644 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:09.890188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:54589 (local address 127.30.23.190:40095)
0809 16:30:09.890323 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:09.890325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:09.890335 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:30:09.890396 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:09.890396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:09.890407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:09.890470 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:09.890833 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.891367 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.892449 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:09.892706 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:09.892807 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:09.895949 (+ 3142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:54589: BlockingRecv error: recv got EOF from 127.30.23.129:54589 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
Aug 09 16:30:10 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:10 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:10.024927 13648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:10.019907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:43263)
0809 16:30:10.020101 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:10.020116 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:10.020224 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:10.020427 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:10.020428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:10.020621 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:10.020670 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:10.020672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:10.021407 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:10.021408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:10.021951 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:10.021953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:10.022327 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:10.022328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:10.022397 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:10.022888 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:10.022900 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:10.024855 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":90,"threads_started":1}
W20260809 16:30:10.025236 13644 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:10.020027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:43263 (local address 127.30.23.190:40095)
0809 16:30:10.020124 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:10.020126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:10.020203 (+ 77us) server_negotiation.cc:407] Connection header received
0809 16:30:10.020307 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:10.020308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:10.020323 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:10.020420 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:10.020754 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:10.021278 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:10.022039 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:10.022239 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:10.022327 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:10.025148 (+ 2821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:43263: BlockingRecv error: recv got EOF from 127.30.23.130:43263 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 16:30:10 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:10 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:10.171370 13650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:10.165709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:58879)
0809 16:30:10.165972 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:10.165982 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:10.166055 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:10.166186 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:10.166187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:10.166363 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:10.166416 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:10.166418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:10.167197 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:10.167198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:10.167852 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:10.167855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:10.168355 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:10.168356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:10.168416 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:10.168921 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:10.168927 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:10.171243 (+ 2316us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":173,"threads_started":1}
W20260809 16:30:10.171406 13644 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:10.165826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:58879 (local address 127.30.23.190:40095)
0809 16:30:10.165913 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:10.165916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:10.166050 (+ 134us) server_negotiation.cc:407] Connection header received
0809 16:30:10.166102 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:10.166103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:10.166115 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:10.166176 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:10.166504 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:10.167082 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:10.168047 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:10.168264 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:10.168332 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:10.171332 (+ 3000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:58879: BlockingRecv error: recv got EOF from 127.30.23.131:58879 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 16:30:10 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:10 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:10.901870 13652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:10.896525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:55799)
0809 16:30:10.896688 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:10.896699 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:10.896757 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:10.896955 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:10.896956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:10.897136 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:10.897190 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:10.897191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:10.897899 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:10.897900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:10.898584 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:10.898588 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:10.898996 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:10.898997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:10.899047 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:10.899540 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:10.899545 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:10.901806 (+ 2261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
W20260809 16:30:10.902172 13653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:10.896607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:55799 (local address 127.30.23.190:40095)
0809 16:30:10.896770 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:10.896772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:10.896806 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:30:10.896832 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:10.896833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:10.896844 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:10.896905 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:10.897267 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:10.897803 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:10.898673 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:10.898912 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:10.898978 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:10.902041 (+ 3063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:55799: BlockingRecv error: recv got EOF from 127.30.23.129:55799 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
Aug 09 16:30:11 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:11 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:11.030356 13654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:11.025533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:56707)
0809 16:30:11.025681 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:11.025692 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:11.025826 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:11.025987 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:11.025988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:11.026174 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:11.026223 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:11.026224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:11.026907 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:11.026908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:11.027429 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:11.027430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:11.027812 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:11.027813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:11.027876 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:11.028387 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:11.028392 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:11.030260 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
W20260809 16:30:11.030426 13653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:11.025681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:56707 (local address 127.30.23.190:40095)
0809 16:30:11.025765 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:11.025767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:11.025813 (+ 46us) server_negotiation.cc:407] Connection header received
0809 16:30:11.025907 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:11.025908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:11.025921 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:11.025971 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:11.026311 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:11.026817 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:11.027512 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:11.027723 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:11.027811 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:11.030364 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:56707: BlockingRecv error: recv got EOF from 127.30.23.130:56707 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 16:30:11 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:11 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:11.177134 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:11.172014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:38101)
0809 16:30:11.172153 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:11.172166 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:11.172239 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:11.172395 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:11.172396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:11.172584 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:11.172638 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:11.172640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:11.173408 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:11.173409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:11.174220 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:11.174224 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:11.174563 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:11.174564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:11.174613 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:11.175169 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:11.175181 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:11.177069 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
W20260809 16:30:11.177258 13653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:11.172181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:38101 (local address 127.30.23.190:40095)
0809 16:30:11.172230 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:11.172232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:11.172244 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:30:11.172294 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:11.172295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:11.172307 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:11.172373 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:11.172714 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:11.173273 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:11.174296 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:11.174486 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:11.174543 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:11.177158 (+ 2615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:38101: BlockingRecv error: recv got EOF from 127.30.23.131:38101 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 16:30:11 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:11 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:11.909176 13658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:11.902501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:56165)
0809 16:30:11.902630 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:11.902641 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:11.902704 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:11.902895 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:11.902896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:11.903123 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:11.903189 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:11.903192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:11.904383 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:11.904384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:11.905445 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:11.905448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:11.906095 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:11.906096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:11.906172 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:11.906808 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:11.906815 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:11.909093 (+ 2278us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
W20260809 16:30:11.909409 13660 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:11.902545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:56165 (local address 127.30.23.190:40095)
0809 16:30:11.902681 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:11.902683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:11.902711 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:30:11.902758 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:11.902759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:11.902774 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:11.902868 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:11.903286 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:11.904266 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:11.905558 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:11.905943 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:11.906020 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:11.909224 (+ 3204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:56165: BlockingRecv error: recv got EOF from 127.30.23.129:56165 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
Aug 09 16:30:12 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:12 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:12.036455 13663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:12.031017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:38329)
0809 16:30:12.031166 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:12.031182 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:12.031249 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:12.031388 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:12.031394 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:12.031577 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:12.031629 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:12.031631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:12.032456 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:12.032457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:12.033063 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:12.033065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:12.033522 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:12.033523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:12.033578 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:12.034149 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:12.034154 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:12.036372 (+ 2218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
W20260809 16:30:12.036592 13660 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:12.031125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:38329 (local address 127.30.23.190:40095)
0809 16:30:12.031179 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:12.031181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:12.031234 (+ 53us) server_negotiation.cc:407] Connection header received
0809 16:30:12.031296 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:12.031297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:12.031308 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:12.031367 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:12.031729 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:12.032342 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:12.033172 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:12.033436 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:12.033506 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:12.036506 (+ 3000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:38329: BlockingRecv error: recv got EOF from 127.30.23.130:38329 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 16:30:12 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:12 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:12.182893 13664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:12.177700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:52007)
0809 16:30:12.177860 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:12.177872 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:12.177947 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:12.178131 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:12.178133 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:12.178309 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:12.178363 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:12.178364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:12.179123 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:12.179124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:12.179664 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:12.179666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:12.180112 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:12.180113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:12.180162 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:12.180629 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:12.180634 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:12.182788 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":114,"threads_started":1}
W20260809 16:30:12.182941 13660 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:12.177780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:52007 (local address 127.30.23.190:40095)
0809 16:30:12.177872 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:12.177874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:12.177923 (+ 49us) server_negotiation.cc:407] Connection header received
0809 16:30:12.178014 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:12.178015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:12.178028 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:12.178076 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:12.178520 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:12.179044 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:12.179791 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:12.180026 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:12.180097 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:12.182873 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:52007: BlockingRecv error: recv got EOF from 127.30.23.131:52007 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 16:30:12 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:12 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:12.919407 13666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:12.909685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:48893)
0809 16:30:12.909847 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:12.909860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:12.909938 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:12.910081 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:12.910082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:12.910268 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:12.910320 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:12.910321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:12.911095 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:12.911096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:12.911672 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:12.911673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:12.912076 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:12.912077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:12.912120 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:12.917051 (+ 4931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:12.917057 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:12.919313 (+ 2256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":109,"threads_started":1}
W20260809 16:30:12.919634 13668 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:12.909810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:48893 (local address 127.30.23.190:40095)
0809 16:30:12.909923 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:12.909924 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:12.909934 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:30:12.910005 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:12.910006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:12.910017 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:12.910076 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:12.910418 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:12.911014 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:12.911779 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:12.912000 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:12.912067 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:12.919543 (+ 7476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:48893: BlockingRecv error: recv got EOF from 127.30.23.129:48893 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":55,"threads_started":1}
Aug 09 16:30:13 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:13 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:13.042135 13669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:13.037041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:41487)
0809 16:30:13.037234 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:13.037245 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:13.037343 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:13.037536 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:13.037537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:13.037779 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:13.037862 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:13.037864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:13.038569 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:13.038571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:13.039112 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:13.039113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:13.039494 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:13.039495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:13.039557 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:13.040136 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:13.040142 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:13.042035 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":99,"threads_started":1}
W20260809 16:30:13.042186 13668 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:13.037106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:41487 (local address 127.30.23.190:40095)
0809 16:30:13.037187 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:13.037190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:13.037328 (+ 138us) server_negotiation.cc:407] Connection header received
0809 16:30:13.037438 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:13.037439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:13.037459 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:13.037528 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:13.037952 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:13.038471 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:13.039199 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:13.039418 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:13.039497 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:13.042119 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:41487: BlockingRecv error: recv got EOF from 127.30.23.130:41487 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 16:30:13 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:13 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:13.188755 13671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:13.183458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:35153)
0809 16:30:13.183649 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:13.183662 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:13.183778 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:13.184000 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:13.184001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:13.184203 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:13.184258 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:13.184260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:13.185062 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:13.185063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:13.185729 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:13.185731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:13.186129 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:13.186130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:13.186195 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:13.186700 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:13.186705 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:13.188619 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":99,"threads_started":1}
W20260809 16:30:13.188762 13668 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:13.183548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:35153 (local address 127.30.23.190:40095)
0809 16:30:13.183630 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:13.183633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:13.183754 (+ 121us) server_negotiation.cc:407] Connection header received
0809 16:30:13.183848 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:13.183849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:13.183867 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:13.183977 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:13.184347 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:13.184963 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:13.185818 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:13.186033 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:13.186130 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:13.188704 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:35153: BlockingRecv error: recv got EOF from 127.30.23.131:35153 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 16:30:13 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:13 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:13.925238 13675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:13.919984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:33693)
0809 16:30:13.920164 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:13.920176 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:13.920240 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:13.920390 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:13.920391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:13.920576 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:13.920629 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:13.920630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:13.921393 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:13.921394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:13.921987 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:13.921989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:13.922511 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:13.922512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:13.922582 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:13.923134 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:13.923139 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:13.925127 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":98,"threads_started":1}
W20260809 16:30:13.925307 13678 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:13.920105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:33693 (local address 127.30.23.190:40095)
0809 16:30:13.920258 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:13.920259 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:13.920284 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:30:13.920308 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:13.920309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:13.920320 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:13.920372 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:13.920696 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:13.921295 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:13.922112 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:13.922445 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:13.922511 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:13.925227 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:33693: BlockingRecv error: recv got EOF from 127.30.23.129:33693 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
Aug 09 16:30:14 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:14 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:14.048419 13679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:14.042788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:36361)
0809 16:30:14.043013 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:14.043030 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:14.043087 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:14.043276 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:14.043277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:14.043456 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:14.043506 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:14.043507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:14.044405 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:14.044406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:14.044943 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:14.044945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:14.045329 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:14.045330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:14.045424 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:14.045967 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:14.045973 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:14.048287 (+ 2314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":86,"threads_started":1}
W20260809 16:30:14.048447 13678 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:14.043023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:36361 (local address 127.30.23.190:40095)
0809 16:30:14.043116 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:14.043118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:14.043127 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:30:14.043155 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:14.043156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:14.043166 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:14.043221 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:14.043623 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:14.044318 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:14.045044 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:14.045246 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:14.045327 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:14.048385 (+ 3058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:36361: BlockingRecv error: recv got EOF from 127.30.23.130:36361 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 16:30:14 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:14 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:14.193925 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:14.189277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:52767)
0809 16:30:14.189493 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:14.189503 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:14.189561 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:14.189729 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:14.189729 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:14.189899 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:14.189949 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:14.189951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:14.190604 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:14.190605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:14.191147 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:14.191149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:14.191500 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:14.191501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:14.191552 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:14.192021 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:14.192025 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:14.193865 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":139,"threads_started":1}
W20260809 16:30:14.194031 13678 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:14.189399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:52767 (local address 127.30.23.190:40095)
0809 16:30:14.189472 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:14.189474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:14.189595 (+ 121us) server_negotiation.cc:407] Connection header received
0809 16:30:14.189633 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:14.189634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:14.189648 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:14.189709 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:14.190016 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:14.190516 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:14.191226 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:14.191404 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:14.191464 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:14.193958 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:52767: BlockingRecv error: recv got EOF from 127.30.23.131:52767 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 16:30:14 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:14 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:14.930975 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:14.925916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:39359)
0809 16:30:14.926085 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:14.926100 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:14.926180 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:14.926363 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:14.926364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:14.926581 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:14.926641 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:14.926643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:14.927345 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:14.927347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:14.927885 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:14.927887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:14.928253 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:14.928254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:14.928312 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:14.928819 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:14.928824 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:14.930912 (+ 2088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
W20260809 16:30:14.931136 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:14.926030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:39359 (local address 127.30.23.190:40095)
0809 16:30:14.926176 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:14.926178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:14.926188 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:30:14.926272 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:14.926273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:14.926287 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:14.926341 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:14.926723 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:14.927234 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:14.927974 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:14.928175 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:14.928262 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:14.931036 (+ 2774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:39359: BlockingRecv error: recv got EOF from 127.30.23.129:39359 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
Aug 09 16:30:15 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:15 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:15.053889 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:15.049046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:50143)
0809 16:30:15.049219 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:15.049233 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:15.049329 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:15.049504 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:15.049506 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:15.049687 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:15.049734 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:15.049736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:15.050414 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:15.050415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:15.050941 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:15.050942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:15.051315 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:15.051316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:15.051385 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:15.051831 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:15.051836 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:15.053827 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
W20260809 16:30:15.054041 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:15.049157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:50143 (local address 127.30.23.190:40095)
0809 16:30:15.049239 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:15.049241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:15.049307 (+ 66us) server_negotiation.cc:407] Connection header received
0809 16:30:15.049425 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:15.049426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:15.049439 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:15.049489 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:15.049812 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:15.050322 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:15.051027 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:15.051231 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:15.051306 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:15.053960 (+ 2654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:50143: BlockingRecv error: recv got EOF from 127.30.23.130:50143 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 16:30:15 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:15 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:15.199491 13686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:15.194460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:59261)
0809 16:30:15.194616 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:15.194626 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:15.194713 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:15.194845 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:15.194846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:15.195021 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:15.195070 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:15.195071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:15.195787 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:15.195788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:15.196326 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:15.196328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:15.196718 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:15.196719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:15.196766 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:15.197319 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:15.197324 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:15.199430 (+ 2106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
W20260809 16:30:15.199672 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:15.194558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:59261 (local address 127.30.23.190:40095)
0809 16:30:15.194634 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:15.194636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:15.194686 (+ 50us) server_negotiation.cc:407] Connection header received
0809 16:30:15.194758 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:15.194758 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:15.194769 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:15.194826 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:15.195167 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:15.195684 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:15.196449 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:15.196639 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:15.196706 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:15.199579 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:59261: BlockingRecv error: recv got EOF from 127.30.23.131:59261 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 16:30:15 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:15 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:15.936787 13687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:15.931581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:34973)
0809 16:30:15.931732 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:15.931742 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:15.931794 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:15.931995 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:15.931996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:15.932164 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:15.932217 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:15.932218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:15.932938 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:15.932939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:15.933550 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:15.933553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:15.933987 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:15.933988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:15.934038 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:15.934687 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:15.934693 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:15.936717 (+ 2024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
W20260809 16:30:15.936908 13688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:15.931662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:34973 (local address 127.30.23.190:40095)
0809 16:30:15.931803 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:15.931806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:15.931815 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:30:15.931847 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:15.931848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:15.931863 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:15.931963 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:15.932318 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:15.932837 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:15.933653 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:15.933881 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:15.933942 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:15.936810 (+ 2868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:34973: BlockingRecv error: recv got EOF from 127.30.23.129:34973 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
Aug 09 16:30:16 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:16 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:16.060004 13691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:16.054491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:48125)
0809 16:30:16.054694 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:16.054724 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:16.054811 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:16.055006 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:16.055007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:16.055206 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:16.055257 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:16.055258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:16.056013 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:16.056014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:16.056553 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:16.056555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:16.056928 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:16.056929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:16.056989 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:16.057623 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:16.057628 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:16.059872 (+ 2244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":84,"threads_started":1}
W20260809 16:30:16.060050 13688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:16.054767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:48125 (local address 127.30.23.190:40095)
0809 16:30:16.054852 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:16.054855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:16.054866 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:30:16.054897 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:16.054899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:16.054916 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:16.054995 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:16.055366 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:16.055918 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:16.056648 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:16.056851 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:16.056929 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:16.059980 (+ 3051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:48125: BlockingRecv error: recv got EOF from 127.30.23.130:48125 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 16:30:16 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:16 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:16.205580 13693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:16.200057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:52615)
0809 16:30:16.200198 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:16.200214 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:16.200289 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:16.200455 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:16.200456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:16.200659 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:16.200721 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:16.200722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:16.201589 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:16.201590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:16.202174 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:16.202177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:16.202650 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:16.202651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:16.202731 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:16.203314 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:16.203319 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:16.205474 (+ 2155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
W20260809 16:30:16.205636 13688 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:16.200190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:52615 (local address 127.30.23.190:40095)
0809 16:30:16.200274 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:16.200277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:16.200289 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:30:16.200368 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:16.200369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:16.200384 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:16.200449 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:16.200821 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:16.201509 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:16.202283 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:16.202565 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:16.202651 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:16.205567 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:52615: BlockingRecv error: recv got EOF from 127.30.23.131:52615 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 16:30:16 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:16 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:16.942461 13694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:16.937291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:59333)
0809 16:30:16.937456 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:16.937468 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:16.937521 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:16.937756 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:16.937757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:16.937939 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:16.937991 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:16.937993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:16.938649 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:16.938650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:16.939185 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:16.939187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:16.939650 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:16.939653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:16.939726 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:16.940310 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:16.940315 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:16.942400 (+ 2085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
W20260809 16:30:16.942634 13695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:16.937499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:59333 (local address 127.30.23.190:40095)
0809 16:30:16.937631 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:16.937632 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:16.937640 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:30:16.937664 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:16.937664 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:16.937674 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:16.937740 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:16.938061 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:16.938571 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:16.939327 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:16.939549 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:16.939640 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:16.942551 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:59333: BlockingRecv error: recv got EOF from 127.30.23.129:59333 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
Aug 09 16:30:17 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:17 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:17.066390 13696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:17.060779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:33267)
0809 16:30:17.060963 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:17.060975 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:17.061029 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:17.061232 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:17.061233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:17.061512 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:17.061576 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:17.061579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:17.062502 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:17.062504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:17.063080 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:17.063083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:17.063532 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:17.063533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:17.063586 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:17.064080 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:17.064084 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:17.066308 (+ 2224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":120,"threads_started":1}
W20260809 16:30:17.066594 13695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:17.060883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:33267 (local address 127.30.23.190:40095)
0809 16:30:17.060984 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:17.060986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:17.061015 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:30:17.061100 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:17.061101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:17.061116 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:17.061204 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:17.061662 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:17.062392 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:17.063184 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:17.063405 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:17.063473 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:17.066517 (+ 3044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:33267: BlockingRecv error: recv got EOF from 127.30.23.130:33267 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 16:30:17 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:17 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:17.210808 13698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:17.206163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:48675)
0809 16:30:17.206325 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:17.206336 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:17.206394 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:17.206528 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:17.206528 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:17.206700 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:17.206750 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:17.206752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:17.207448 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:17.207449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:17.207995 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:17.207997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:17.208353 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:17.208354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:17.208396 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:17.208879 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:17.208884 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:17.210745 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
W20260809 16:30:17.210969 13695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:17.206263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:48675 (local address 127.30.23.190:40095)
0809 16:30:17.206321 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:17.206323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:17.206377 (+ 54us) server_negotiation.cc:407] Connection header received
0809 16:30:17.206453 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:17.206454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:17.206466 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:17.206508 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:17.206852 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:17.207362 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:17.208081 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:17.208285 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:17.208343 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:17.210888 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:48675: BlockingRecv error: recv got EOF from 127.30.23.131:48675 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 16:30:17 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:17 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:17.951097 13699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:17.942951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:44061)
0809 16:30:17.943083 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:17.943095 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:17.943165 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:17.943361 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:17.943363 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:17.943556 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:17.943615 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:17.943616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:17.944377 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:17.944378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:17.944982 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:17.944984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:17.945445 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:17.945447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:17.945496 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:17.949027 (+ 3531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:17.949032 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:17.951035 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
W20260809 16:30:17.951334 13700 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:17.942997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:44061 (local address 127.30.23.190:40095)
0809 16:30:17.943152 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:17.943154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:17.943163 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:30:17.943224 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:17.943225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:17.943238 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:17.943292 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:17.943709 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:17.944292 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:17.945084 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:17.945306 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:17.945452 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:17.951248 (+ 5796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:44061: BlockingRecv error: recv got EOF from 127.30.23.129:44061 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
Aug 09 16:30:18 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:18 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:18.071990 13702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:18.066874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:60931)
0809 16:30:18.067037 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:18.067048 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:18.067103 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:18.067275 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:18.067276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:18.067477 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:18.067530 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:18.067532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:18.068255 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:18.068256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:18.068793 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:18.068795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:18.069195 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:18.069196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:18.069265 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:18.069861 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:18.069866 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:18.071928 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
W20260809 16:30:18.072188 13700 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:18.066987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:60931 (local address 127.30.23.190:40095)
0809 16:30:18.067056 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:18.067058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:18.067092 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:30:18.067157 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:18.067158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:18.067168 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:18.067222 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:18.067651 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:18.068161 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:18.068906 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:18.069100 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:18.069167 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:18.072105 (+ 2938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:60931: BlockingRecv error: recv got EOF from 127.30.23.130:60931 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 16:30:18 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:18 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:18.216097 13703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:18.211269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:46761)
0809 16:30:18.211454 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:18.211464 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:18.211521 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:18.211668 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:18.211669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:18.211844 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:18.211893 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:18.211894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:18.212569 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:18.212570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:18.213135 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:18.213138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:18.213573 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:18.213574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:18.213627 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:18.214089 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:18.214094 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:18.216024 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":98,"threads_started":1}
W20260809 16:30:18.216163 13700 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:18.211353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:46761 (local address 127.30.23.190:40095)
0809 16:30:18.211419 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:18.211421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:18.211523 (+ 102us) server_negotiation.cc:407] Connection header received
0809 16:30:18.211591 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:18.211592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:18.211605 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:18.211650 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:18.211957 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:18.212464 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:18.213231 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:18.213486 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:18.213553 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:18.216116 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:46761: BlockingRecv error: recv got EOF from 127.30.23.131:46761 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 16:30:18 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:18 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:18.956547 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:18.951618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:49353)
0809 16:30:18.951879 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:18.951891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:18.951959 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:18.952090 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:18.952091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:18.952268 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:18.952320 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:18.952321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:18.952988 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:18.952989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:18.953625 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:18.953627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:18.954003 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:18.954004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:18.954045 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:18.954592 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:18.954597 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:18.956485 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":193,"threads_started":1}
W20260809 16:30:18.956657 13705 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:18.951826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:49353 (local address 127.30.23.190:40095)
0809 16:30:18.951973 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:18.951976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:18.952000 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:30:18.952021 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:18.952022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:18.952033 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:18.952084 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:18.952385 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:18.952904 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:18.953705 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:18.953922 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:18.954001 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:18.956582 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:49353: BlockingRecv error: recv got EOF from 127.30.23.129:49353 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
Aug 09 16:30:19 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:19 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:19.077482 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:19.072573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:44743)
0809 16:30:19.072736 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:19.072746 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:19.072818 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:19.073009 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:19.073010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:19.073189 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:19.073241 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:19.073243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:19.073998 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:19.073999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:19.074522 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:19.074524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:19.074954 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:19.074955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:19.075009 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:19.075493 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:19.075498 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:19.077410 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
W20260809 16:30:19.077523 13705 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:19.072688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:44743 (local address 127.30.23.190:40095)
0809 16:30:19.072784 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:19.072786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:19.072795 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:30:19.072885 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:19.072886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:19.072897 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:19.072955 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:19.073331 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:19.073901 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:19.074624 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:19.074856 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:19.074923 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:19.077475 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:44743: BlockingRecv error: recv got EOF from 127.30.23.130:44743 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 16:30:19 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:19 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:19.221565 13708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:19.216634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:33273)
0809 16:30:19.216835 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:19.216847 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:19.216911 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:19.217040 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:19.217040 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:19.217212 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:19.217262 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:19.217263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:19.217998 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:19.217999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:19.218519 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:19.218520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:19.218881 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:19.218882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:19.218939 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:19.219435 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:19.219440 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:19.221502 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":104,"threads_started":1}
W20260809 16:30:19.221728 13705 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:19.216780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:33273 (local address 127.30.23.190:40095)
0809 16:30:19.216854 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:19.216856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:19.216918 (+ 62us) server_negotiation.cc:407] Connection header received
0809 16:30:19.216962 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:19.216962 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:19.216975 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:19.217034 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:19.217367 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:19.217877 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:19.218603 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:19.218791 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:19.218851 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:19.221630 (+ 2779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:33273: BlockingRecv error: recv got EOF from 127.30.23.131:33273 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 16:30:19 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:19 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:19.962416 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:19.957115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:47283)
0809 16:30:19.957290 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:19.957304 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:19.957400 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:19.957658 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:19.957659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:19.957854 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:19.957927 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:19.957929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:19.958692 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:19.958693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:19.959310 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:19.959312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:19.959732 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:19.959733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:19.959810 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:19.960401 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:19.960406 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:19.962344 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":89,"threads_started":1}
W20260809 16:30:19.962613 13712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:19.957280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:47283 (local address 127.30.23.190:40095)
0809 16:30:19.957473 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:19.957475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:19.957484 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:30:19.957509 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:19.957510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:19.957522 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:19.957603 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:19.958045 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:19.958560 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:19.959432 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:19.959645 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:19.959710 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:19.962528 (+ 2818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:47283: BlockingRecv error: recv got EOF from 127.30.23.129:47283 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":93,"threads_started":1}
Aug 09 16:30:20 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:20 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:20.082867 13713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:20.078006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:59307)
0809 16:30:20.078233 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:20.078260 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:20.078357 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:20.078517 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:20.078518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:20.078720 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:20.078770 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:20.078772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:20.079449 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:20.079450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:20.080000 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:20.080001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:20.080363 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:20.080364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:20.080425 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:20.080950 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:20.080956 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:20.082756 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":117,"threads_started":1}
W20260809 16:30:20.082913 13712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:20.078102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:59307 (local address 127.30.23.190:40095)
0809 16:30:20.078217 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:20.078219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:20.078337 (+ 118us) server_negotiation.cc:407] Connection header received
0809 16:30:20.078419 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:20.078420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:20.078434 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:20.078507 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:20.078854 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:20.079358 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:20.080094 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:20.080291 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:20.080363 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:20.082846 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:59307: BlockingRecv error: recv got EOF from 127.30.23.130:59307 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 16:30:20 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:20 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:20.227326 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:20.222123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:41425)
0809 16:30:20.222276 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:20.222287 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:20.222378 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:20.222555 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:20.222556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:20.222731 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:20.222778 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:20.222780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:20.223476 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:20.223477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:20.224037 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:20.224039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:20.224411 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:20.224411 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:20.224455 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:20.225086 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:20.225091 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:20.227235 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
W20260809 16:30:20.227399 13712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:20.222173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:41425 (local address 127.30.23.190:40095)
0809 16:30:20.222231 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:20.222233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:20.222360 (+ 127us) server_negotiation.cc:407] Connection header received
0809 16:30:20.222455 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:20.222456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:20.222479 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:20.222526 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:20.222882 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:20.223395 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:20.224132 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:20.224338 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:20.224401 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:20.227338 (+ 2937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:41425: BlockingRecv error: recv got EOF from 127.30.23.131:41425 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 09 16:30:20 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:20 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:20.968145 13717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:20.963023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:46247)
0809 16:30:20.963173 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:20.963187 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:20.963273 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:20.963471 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:20.963472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:20.963717 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:20.963777 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:20.963780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:20.964480 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:20.964481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:20.965020 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:20.965022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:20.965416 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:20.965417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:20.965461 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:20.966076 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:20.966081 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:20.968082 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
W20260809 16:30:20.968261 13718 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:20.963129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:46247 (local address 127.30.23.190:40095)
0809 16:30:20.963241 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:20.963243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:20.963253 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:30:20.963355 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:20.963355 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:20.963370 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:20.963460 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:20.963852 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:20.964355 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:20.965107 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:20.965302 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:20.965433 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:20.968185 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:46247: BlockingRecv error: recv got EOF from 127.30.23.129:46247 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":54,"threads_started":1}
Aug 09 16:30:21 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:21 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:21.088938 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:21.083456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:43561)
0809 16:30:21.083667 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:21.083680 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:21.083782 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:21.083937 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:21.083938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:21.084143 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:21.084190 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:21.084191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:21.085018 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:21.085020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:21.085862 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:21.085865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:21.086291 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:21.086292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:21.086345 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:21.086900 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:21.086905 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:21.088877 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":110,"threads_started":1}
W20260809 16:30:21.089083 13718 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:21.083586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:43561 (local address 127.30.23.190:40095)
0809 16:30:21.083666 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:21.083668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:21.083760 (+ 92us) server_negotiation.cc:407] Connection header received
0809 16:30:21.083855 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:21.083856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:21.083870 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:21.083921 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:21.084298 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:21.084883 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:21.085967 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:21.086186 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:21.086251 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:21.088999 (+ 2748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:43561: BlockingRecv error: recv got EOF from 127.30.23.130:43561 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 16:30:21 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:21 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:21.233101 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:21.228043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:52957)
0809 16:30:21.228342 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:21.228355 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:21.228441 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:21.228657 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:21.228658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:21.228985 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:21.229048 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:21.229051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:21.229829 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:21.229830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:21.230368 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:21.230370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:21.230742 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:21.230743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:21.230788 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:21.231270 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:21.231274 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:21.233042 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":111,"threads_started":1}
W20260809 16:30:21.233202 13718 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:21.228155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:52957 (local address 127.30.23.190:40095)
0809 16:30:21.228346 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:21.228349 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:21.228429 (+ 80us) server_negotiation.cc:407] Connection header received
0809 16:30:21.228514 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:21.228515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:21.228537 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:21.228610 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:21.229118 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:21.229752 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:21.230478 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:21.230663 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:21.230728 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:21.233136 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:52957: BlockingRecv error: recv got EOF from 127.30.23.131:52957 (error 108)
Metrics: {"server-negotiator.queue_time_us":156}
Aug 09 16:30:21 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:21 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:21.975752 13723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:21.968684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:48235)
0809 16:30:21.968840 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:21.968855 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:21.968953 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:21.969111 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:21.969113 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:21.969424 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:21.969498 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:21.969501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:21.970616 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:21.970617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:21.971738 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:21.971741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:21.972249 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:21.972250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:21.972347 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:21.972965 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:21.972972 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:21.975605 (+ 2633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
W20260809 16:30:21.975790 13724 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:21.968748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:48235 (local address 127.30.23.190:40095)
0809 16:30:21.968882 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:21.968884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:21.968918 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:30:21.969029 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:21.969030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:21.969041 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:21.969098 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:21.969635 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:21.970514 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:21.971830 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:21.972155 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:21.972227 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:21.975716 (+ 3489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:48235: BlockingRecv error: recv got EOF from 127.30.23.129:48235 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":54,"threads_started":1}
Aug 09 16:30:22 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:22 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:22.094985 13725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:22.089674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:51141)
0809 16:30:22.089899 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:22.089912 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:22.089986 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:22.090150 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:22.090151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:22.090350 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:22.090402 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:22.090404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:22.091103 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:22.091104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:22.091658 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:22.091660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:22.092041 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:22.092042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:22.092110 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:22.092640 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:22.092645 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:22.094846 (+ 2201us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":124,"threads_started":1}
W20260809 16:30:22.095039 13724 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:22.089795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:51141 (local address 127.30.23.190:40095)
0809 16:30:22.089848 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:22.089851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:22.089989 (+ 138us) server_negotiation.cc:407] Connection header received
0809 16:30:22.090047 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:22.090048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:22.090062 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:22.090136 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:22.090497 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:22.091015 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:22.091755 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:22.091963 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:22.092041 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:22.094991 (+ 2950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:51141: BlockingRecv error: recv got EOF from 127.30.23.130:51141 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 16:30:22 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:22 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:22.239477 13728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:22.233751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:58875)
0809 16:30:22.233903 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:22.233929 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:22.234007 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:22.234309 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:22.234311 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:22.234685 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:22.234778 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:22.234780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:22.235610 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:22.235611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:22.236166 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:22.236168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:22.236585 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:22.236585 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:22.236685 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:22.237183 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:22.237188 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:22.239404 (+ 2216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
W20260809 16:30:22.239599 13724 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:22.233882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:58875 (local address 127.30.23.190:40095)
0809 16:30:22.233949 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:22.233951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:22.233985 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:30:22.234137 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:22.234139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:22.234157 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:22.234259 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:22.234909 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:22.235518 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:22.236278 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:22.236504 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:22.236569 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:22.239516 (+ 2947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:58875: BlockingRecv error: recv got EOF from 127.30.23.131:58875 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 16:30:22 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:22 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:22.984988 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:22.976368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:57129)
0809 16:30:22.976516 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:22.976528 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:22.976585 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:22.976762 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:22.976763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:22.976945 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:22.976993 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:22.976994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:22.977669 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:22.977670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:22.978210 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:22.978212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:22.978607 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:22.978608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:22.978653 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:22.982880 (+ 4227us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:22.982886 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:22.984888 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
W20260809 16:30:22.985054 13731 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:22.976490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:57129 (local address 127.30.23.190:40095)
0809 16:30:22.976626 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:22.976628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:22.976636 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:30:22.976667 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:22.976668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:22.976678 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:22.976745 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:22.977065 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:22.977594 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:22.978319 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:22.978526 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:22.978592 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:22.984983 (+ 6391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:57129: BlockingRecv error: recv got EOF from 127.30.23.129:57129 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
Aug 09 16:30:23 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:23 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:23.100718 13732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:23.095608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:58235)
0809 16:30:23.095922 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:23.095934 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:23.096007 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:23.096164 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:23.096165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:23.096370 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:23.096417 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:23.096420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:23.097100 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:23.097102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:23.097682 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:23.097684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:23.098208 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:23.098209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:23.098279 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:23.098773 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:23.098777 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:23.100658 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":127,"threads_started":1}
W20260809 16:30:23.100952 13731 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:23.095712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:58235 (local address 127.30.23.190:40095)
0809 16:30:23.095905 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:23.095907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:23.095986 (+ 79us) server_negotiation.cc:407] Connection header received
0809 16:30:23.096065 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:23.096066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:23.096079 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:23.096157 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:23.096502 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:23.097010 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:23.097785 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:23.098078 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:23.098205 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:23.100888 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:58235: BlockingRecv error: recv got EOF from 127.30.23.130:58235 (error 108)
Metrics: {"server-negotiator.queue_time_us":157}
Aug 09 16:30:23 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:23 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.131@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:23.245216 13734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:23.240086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.131:48809)
0809 16:30:23.240242 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:23.240253 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:23.240320 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:23.240557 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:23.240558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:23.240743 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:23.240791 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:23.240792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:23.241639 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:23.241640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:23.242207 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:23.242209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:23.242592 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:23.242593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:23.242644 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:23.243086 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:23.243090 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:23.245091 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
W20260809 16:30:23.245244 13731 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:23.240142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:48809 (local address 127.30.23.190:40095)
0809 16:30:23.240210 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:23.240212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:23.240377 (+ 165us) server_negotiation.cc:407] Connection header received
0809 16:30:23.240408 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:23.240409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:23.240424 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:23.240513 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:23.240927 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:23.241545 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:23.242289 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:23.242503 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:23.242561 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:23.245174 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.131:48809: BlockingRecv error: recv got EOF from 127.30.23.131:48809 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 16:30:23 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:23 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.129@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:23.990934 13735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:23.985740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.129:50029)
0809 16:30:23.985931 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:23.985945 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:23.986023 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:23.986258 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:23.986259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:23.986461 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:23.986516 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:23.986517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:23.987225 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:23.987226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:23.987788 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:23.987791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:23.988170 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:23.988171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:23.988218 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:23.988752 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:23.988757 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:23.990872 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":119,"threads_started":1}
W20260809 16:30:23.991119 13736 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:23.985889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:50029 (local address 127.30.23.190:40095)
0809 16:30:23.986066 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:23.986068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:23.986078 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:30:23.986109 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:23.986109 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:23.986120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:23.986184 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:23.986610 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:23.987135 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:23.987889 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:23.988094 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:23.988158 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:23.991021 (+ 2863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.129:50029: BlockingRecv error: recv got EOF from 127.30.23.129:50029 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
Aug 09 16:30:24 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:30:24 dist-test-slave-j2vl krb5kdc[11949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.23.130@KRBTEST.COM for oryx/127.30.23.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:30:24.108925 13739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:24.101466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40095 (local address 127.30.23.130:38583)
0809 16:30:24.101740 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:24.101759 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:24.101869 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:24.102083 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:24.102084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:24.102400 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:24.102477 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:24.102480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.103587 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.103588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:24.104399 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:24.104403 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.104931 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.104932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:24.105002 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:24.105761 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:24.105771 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:24.108760 (+ 2989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.23.190:40095: Server oryx/127.30.23.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":142,"threads_started":1}
W20260809 16:30:24.108996 13736 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:24.101599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:38583 (local address 127.30.23.190:40095)
0809 16:30:24.101682 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:24.101684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:24.101854 (+ 170us) server_negotiation.cc:407] Connection header received
0809 16:30:24.101972 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:24.101972 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:24.101989 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:24.102067 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:24.102588 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.103446 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.104560 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.104814 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.104909 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:24.108898 (+ 3989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.23.130:38583: BlockingRecv error: recv got EOF from 127.30.23.130:38583 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:30:24.154120 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 12947
I20260809 16:30:24.176925 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 13212
I20260809 16:30:24.189579 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 13436
I20260809 16:30:24.204203 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 12820
2026-08-09T16:30:24Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17891 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T16:30:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:30:24Z Disabled control of system clock
I20260809 16:30:24.259117 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:34555
--webserver_interface=127.30.23.190
--webserver_port=0
--builtin_ntp_servers=127.30.23.148:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:34555
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 16:30:24.387899 13743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:24.388234 13743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:24.388266 13743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:24.389904 13743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:30:24.389981 13743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:24.389998 13743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:24.390022 13743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:30:24.390040 13743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:30:24.391873 13743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:34555
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:34555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:24.392259 13743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:24.392684 13743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:30:24.400167 13743 server_base.cc:1034] running on GCE node
W20260809 16:30:24.400116 13749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:24.400122 13751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:24.400116 13753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:24.400914 13743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:24.401305 13743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:24.402560 13743 hybrid_clock.cc:648] HybridClock initialized: now 1786293024402531 us; error 44 us; skew 500 ppm
I20260809 16:30:24.404335 13743 webserver.cc:460] Webserver started at http://127.30.23.190:43493/ using document root <none> and password file <none>
I20260809 16:30:24.404659 13743 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:24.404740 13743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:24.404922 13743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:24.406602 13743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "5699cffffb4b4273a455fb00f95e7a70"
format_stamp: "Formatted at 2026-08-09 16:30:24 on dist-test-slave-j2vl"
server_key: "8b49278302c08004569a041f6a63a638"
server_key_iv: "ba57f94ed9ac17c3d48dbb17ed9ff85e"
server_key_version: "encryptionkey@0"
I20260809 16:30:24.407169 13743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "5699cffffb4b4273a455fb00f95e7a70"
format_stamp: "Formatted at 2026-08-09 16:30:24 on dist-test-slave-j2vl"
server_key: "8b49278302c08004569a041f6a63a638"
server_key_iv: "ba57f94ed9ac17c3d48dbb17ed9ff85e"
server_key_version: "encryptionkey@0"
I20260809 16:30:24.410697 13743 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 16:30:24.412719 13765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:24.413216 13743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:30:24.413516 13743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "5699cffffb4b4273a455fb00f95e7a70"
format_stamp: "Formatted at 2026-08-09 16:30:24 on dist-test-slave-j2vl"
server_key: "8b49278302c08004569a041f6a63a638"
server_key_iv: "ba57f94ed9ac17c3d48dbb17ed9ff85e"
server_key_version: "encryptionkey@0"
I20260809 16:30:24.413666 13743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:24.432610 13743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:24.433183 13743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:24.433467 13743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:24.440344 13743 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:34555
I20260809 16:30:24.440409 13848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:34555 every 8 connection(s)
I20260809 16:30:24.441037 13743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:30:24.442166 13849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:24.446498 13849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70: Bootstrap starting.
I20260809 16:30:24.447433 13849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:24.447860 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 13743
I20260809 16:30:24.448040 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:30:24.448040 13849 log.cc:824] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70: Log is configured to *not* fsync() on all Append() calls
I20260809 16:30:24.448436 30814 external_mini_cluster.cc:1428] Setting key a1630da928eaaa2e7cb02e3540498c12
I20260809 16:30:24.449584 13849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70: No bootstrap required, opened a new log
I20260809 16:30:24.452708 13849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5699cffffb4b4273a455fb00f95e7a70" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34555 } }
I20260809 16:30:24.452982 13849 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:24.453035 13849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5699cffffb4b4273a455fb00f95e7a70, State: Initialized, Role: FOLLOWER
I20260809 16:30:24.453207 13849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5699cffffb4b4273a455fb00f95e7a70" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34555 } }
I20260809 16:30:24.453333 13849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:30:24.453512 13849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:30:24.453624 13849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:24.455253 13849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5699cffffb4b4273a455fb00f95e7a70" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34555 } }
I20260809 16:30:24.455461 13849 leader_election.cc:302] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5699cffffb4b4273a455fb00f95e7a70; no voters:
I20260809 16:30:24.455739 13849 leader_election.cc:288] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:30:24.455824 13858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:30:24.456266 13858 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [term 1 LEADER]: Becoming Leader. State: Replica: 5699cffffb4b4273a455fb00f95e7a70, State: Running, Role: LEADER
I20260809 16:30:24.456396 13858 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5699cffffb4b4273a455fb00f95e7a70" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34555 } }
I20260809 16:30:24.456521 13856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:24.449407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53732 (local address 127.30.23.190:34555)
0809 16:30:24.449682 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:24.449691 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:24.449729 (+ 38us) server_negotiation.cc:407] Connection header received
0809 16:30:24.450322 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:24.450335 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:24.450426 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:24.450590 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:24.451047 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.452082 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.453469 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.454169 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.454298 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:24.455439 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:24.455444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:24.455444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:30:24.455727 (+ 283us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:24.455789 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:30:24.455809 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:24.455878 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:24.456156 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 16:30:24.456259 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":124,"threads_started":1}
I20260809 16:30:24.456818 13858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5699cffffb4b4273a455fb00f95e7a70. Latest consensus state: current_term: 1 leader_uuid: "5699cffffb4b4273a455fb00f95e7a70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5699cffffb4b4273a455fb00f95e7a70" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34555 } } }
I20260809 16:30:24.456931 13858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [sys.catalog]: This master's current role is: LEADER
I20260809 16:30:24.456252 13849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:30:24.457212 13862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:30:24.456934 13860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5699cffffb4b4273a455fb00f95e7a70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5699cffffb4b4273a455fb00f95e7a70" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34555 } } }
I20260809 16:30:24.457576 13860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70 [sys.catalog]: This master's current role is: LEADER
I20260809 16:30:24.458050 13862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:30:24.460323 13862 catalog_manager.cc:1347] Generated new cluster ID: 806b093955964486a4c3edb1d7c3faba
I20260809 16:30:24.460431 13862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:30:24.474733 13862 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:30:24.475659 13862 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:30:24.489795 13862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5699cffffb4b4273a455fb00f95e7a70: Generated new TSK 0
I20260809 16:30:24.490252 13862 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:30:24.498646 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34555
--builtin_ntp_servers=127.30.23.148:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 16:30:24.664141 13886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:24.664570 13886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:24.664638 13886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:24.666739 13886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:24.666823 13886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:24.666882 13886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:30:24.668954 13886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:24.669272 13886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:24.669653 13886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:24.677533 13892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:24.677528 13898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:24.677542 13893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:24.678053 13886 server_base.cc:1034] running on GCE node
I20260809 16:30:24.678285 13886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:24.678679 13886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:24.679952 13886 hybrid_clock.cc:648] HybridClock initialized: now 1786293024679909 us; error 34 us; skew 500 ppm
I20260809 16:30:24.681849 13886 webserver.cc:460] Webserver started at http://127.30.23.129:44665/ using document root <none> and password file <none>
I20260809 16:30:24.682168 13886 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:24.682247 13886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:24.682408 13886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:24.683578 13886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "c5ed2f56d0c04c6bb87bee5adcf03c90"
format_stamp: "Formatted at 2026-08-09 16:30:24 on dist-test-slave-j2vl"
server_key: "c98476cd56b980b397d62ab0e554b81a"
server_key_iv: "1f1c257ee567dda8afb716243c5d31c2"
server_key_version: "encryptionkey@0"
I20260809 16:30:24.683969 13886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "c5ed2f56d0c04c6bb87bee5adcf03c90"
format_stamp: "Formatted at 2026-08-09 16:30:24 on dist-test-slave-j2vl"
server_key: "c98476cd56b980b397d62ab0e554b81a"
server_key_iv: "1f1c257ee567dda8afb716243c5d31c2"
server_key_version: "encryptionkey@0"
I20260809 16:30:24.685582 13886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:30:24.686437 13905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:24.686653 13886 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:24.686753 13886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "c5ed2f56d0c04c6bb87bee5adcf03c90"
format_stamp: "Formatted at 2026-08-09 16:30:24 on dist-test-slave-j2vl"
server_key: "c98476cd56b980b397d62ab0e554b81a"
server_key_iv: "1f1c257ee567dda8afb716243c5d31c2"
server_key_version: "encryptionkey@0"
I20260809 16:30:24.686847 13886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:24.712453 13886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:24.712898 13886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:24.713089 13886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:24.713424 13886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:24.713838 13886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:24.713901 13886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:24.713943 13886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:24.713994 13886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:24.721298 13886 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:44537
I20260809 16:30:24.721414 14097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:44537 every 8 connection(s)
I20260809 16:30:24.721781 13886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:30:24.729000 13856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:24.722751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:35133 (local address 127.30.23.190:34555)
0809 16:30:24.722839 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:24.722842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:24.723480 (+ 638us) server_negotiation.cc:407] Connection header received
0809 16:30:24.724021 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:24.724022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:24.724039 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:24.724115 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:24.725203 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.725776 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.726894 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.727108 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.727189 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:24.728687 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:24.728688 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:24.728688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:30:24.728694 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:24.728713 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:30:24.728717 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:24.728775 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:24.728902 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 16:30:24.728930 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:30:24.729380 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:24.722717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34555 (local address 127.30.23.129:35133)
0809 16:30:24.723278 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:24.723331 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:24.723868 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:24.724266 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:24.724270 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:24.724281 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:24.724898 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:24.724904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.725898 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.725899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:24.726760 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:24.726763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.727223 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.727224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:24.727358 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:24.728383 (+ 1025us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:30:24.728384 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:24.728439 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:30:24.728483 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:24.728810 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:24.728817 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:24.728821 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:30:24.728898 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:30:24.729007 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":115,"threads_started":1}
I20260809 16:30:24.730074 14104 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34555
I20260809 16:30:24.730193 14104 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:24.730455 14104 heartbeater.cc:511] Master 127.30.23.190:34555 requested a full tablet report, sending...
I20260809 16:30:24.731005 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 13886
I20260809 16:30:24.731148 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:30:24.731336 13795 ts_manager.cc:194] Registered new tserver with Master: c5ed2f56d0c04c6bb87bee5adcf03c90 (127.30.23.129:44537)
I20260809 16:30:24.731545 30814 external_mini_cluster.cc:1428] Setting key e3ae5ce77c93aa99bdfc009acf7e9230
I20260809 16:30:24.732228 13795 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.23.129:35133
I20260809 16:30:24.733155 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34555
--builtin_ntp_servers=127.30.23.148:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 16:30:24.831408 14115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:24.831645 14115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:24.831668 14115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:24.832860 14115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:24.832906 14115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:24.832931 14115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:30:24.834277 14115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:24.834510 14115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:24.834807 14115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:24.841008 14132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:24.841043 14127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:24.841010 14124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:24.841410 14115 server_base.cc:1034] running on GCE node
I20260809 16:30:24.841681 14115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:24.841958 14115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:24.843158 14115 hybrid_clock.cc:648] HybridClock initialized: now 1786293024843144 us; error 33 us; skew 500 ppm
I20260809 16:30:24.844632 14115 webserver.cc:460] Webserver started at http://127.30.23.130:34203/ using document root <none> and password file <none>
I20260809 16:30:24.844909 14115 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:24.844985 14115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:24.845142 14115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:24.846352 14115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "4b632099a8ba4867a1dfb4318147b8c5"
format_stamp: "Formatted at 2026-08-09 16:30:24 on dist-test-slave-j2vl"
server_key: "8eea9a56e5819d9769242ece6e0cea88"
server_key_iv: "76447d7745d608b62d915376215739b3"
server_key_version: "encryptionkey@0"
I20260809 16:30:24.846750 14115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "4b632099a8ba4867a1dfb4318147b8c5"
format_stamp: "Formatted at 2026-08-09 16:30:24 on dist-test-slave-j2vl"
server_key: "8eea9a56e5819d9769242ece6e0cea88"
server_key_iv: "76447d7745d608b62d915376215739b3"
server_key_version: "encryptionkey@0"
I20260809 16:30:24.848174 14115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:30:24.849529 14140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:24.849764 14115 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:24.849911 14115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "4b632099a8ba4867a1dfb4318147b8c5"
format_stamp: "Formatted at 2026-08-09 16:30:24 on dist-test-slave-j2vl"
server_key: "8eea9a56e5819d9769242ece6e0cea88"
server_key_iv: "76447d7745d608b62d915376215739b3"
server_key_version: "encryptionkey@0"
I20260809 16:30:24.850000 14115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:24.875145 14115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:24.875651 14115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:24.875840 14115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:24.876130 14115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:24.876515 14115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:24.876573 14115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:24.876618 14115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:24.876650 14115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:24.883183 14115 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:33741
I20260809 16:30:24.883232 14321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:33741 every 8 connection(s)
I20260809 16:30:24.883630 14115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:30:24.889236 13856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:24.884485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:56667 (local address 127.30.23.190:34555)
0809 16:30:24.884546 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:24.884548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:24.884780 (+ 232us) server_negotiation.cc:407] Connection header received
0809 16:30:24.885324 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:24.885325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:24.885401 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:24.885470 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:24.886165 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.886678 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.887807 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.888012 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.888077 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:24.889028 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:24.889029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:24.889029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:30:24.889033 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:24.889046 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:30:24.889050 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:24.889093 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:24.889152 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 16:30:24.889175 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:30:24.889514 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:24.884442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34555 (local address 127.30.23.130:56667)
0809 16:30:24.884640 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:24.884683 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:24.885179 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:24.885539 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:24.885542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:24.885551 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:24.886074 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:24.886079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.886806 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.886807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:24.887692 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:24.887694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:24.888118 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:24.888120 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:24.888242 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:24.888888 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:30:24.888889 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:24.888919 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:30:24.888944 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:24.889094 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:24.889096 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:24.889098 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:30:24.889146 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 16:30:24.889227 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260809 16:30:24.890039 14322 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34555
I20260809 16:30:24.890156 14322 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:24.890360 14322 heartbeater.cc:511] Master 127.30.23.190:34555 requested a full tablet report, sending...
I20260809 16:30:24.890893 13794 ts_manager.cc:194] Registered new tserver with Master: 4b632099a8ba4867a1dfb4318147b8c5 (127.30.23.130:33741)
I20260809 16:30:24.891036 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 14115
I20260809 16:30:24.891155 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:30:24.891362 30814 external_mini_cluster.cc:1428] Setting key a4c0b07ccfabb7bd430e04e44426c0a2
I20260809 16:30:24.891525 13794 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.23.130:56667
I20260809 16:30:24.892398 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34555
--builtin_ntp_servers=127.30.23.148:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 16:30:24.982656 14327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:24.982877 14327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:24.982901 14327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:24.984030 14327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:24.984067 14327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:24.984093 14327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:30:24.985419 14327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:24.985628 14327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:24.985899 14327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:24.991611 14337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:24.991616 14339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:24.991611 14342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:24.991904 14327 server_base.cc:1034] running on GCE node
I20260809 16:30:24.992228 14327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:24.992446 14327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:24.993620 14327 hybrid_clock.cc:648] HybridClock initialized: now 1786293024993596 us; error 41 us; skew 500 ppm
I20260809 16:30:24.995180 14327 webserver.cc:460] Webserver started at http://127.30.23.131:43511/ using document root <none> and password file <none>
I20260809 16:30:24.995507 14327 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:24.995558 14327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:24.995656 14327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:24.996693 14327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "f9b74b6cab3c43c2bd3dd8d817d5fe5a"
format_stamp: "Formatted at 2026-08-09 16:30:24 on dist-test-slave-j2vl"
server_key: "8c7332a5599bff4e67f30473de250598"
server_key_iv: "5922ee13c2e02071ac03e8794581b6db"
server_key_version: "encryptionkey@0"
I20260809 16:30:24.997000 14327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "f9b74b6cab3c43c2bd3dd8d817d5fe5a"
format_stamp: "Formatted at 2026-08-09 16:30:24 on dist-test-slave-j2vl"
server_key: "8c7332a5599bff4e67f30473de250598"
server_key_iv: "5922ee13c2e02071ac03e8794581b6db"
server_key_version: "encryptionkey@0"
I20260809 16:30:24.998440 14327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:30:24.999275 14358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:24.999513 14327 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:30:24.999632 14327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "f9b74b6cab3c43c2bd3dd8d817d5fe5a"
format_stamp: "Formatted at 2026-08-09 16:30:24 on dist-test-slave-j2vl"
server_key: "8c7332a5599bff4e67f30473de250598"
server_key_iv: "5922ee13c2e02071ac03e8794581b6db"
server_key_version: "encryptionkey@0"
I20260809 16:30:24.999713 14327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:25.029070 14327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:25.029505 14327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:25.029676 14327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:25.029966 14327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:25.030344 14327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:25.030402 14327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:25.030447 14327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:25.030476 14327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:25.036936 14327 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:38139
I20260809 16:30:25.036964 14536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:38139 every 8 connection(s)
I20260809 16:30:25.037462 14327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:30:25.038939 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 14327
I20260809 16:30:25.039021 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:30:25.039206 30814 external_mini_cluster.cc:1428] Setting key a659188f73b1d5644dd92e59f40f2fb2
I20260809 16:30:25.043839 13856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:25.038474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:40335 (local address 127.30.23.190:34555)
0809 16:30:25.038542 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:25.038544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:25.038821 (+ 277us) server_negotiation.cc:407] Connection header received
0809 16:30:25.039540 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:25.039542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:25.039555 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:25.039642 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:25.040658 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:25.041165 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:25.042394 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:25.042587 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:25.042655 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:25.043592 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:25.043594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:25.043594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:30:25.043599 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:25.043615 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:30:25.043619 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:25.043674 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:25.043725 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 16:30:25.043750 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:30:25.044045 14540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:25.038442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34555 (local address 127.30.23.131:40335)
0809 16:30:25.038693 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:25.038730 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:25.039393 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:25.039750 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:25.039754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:25.039765 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:25.040551 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:25.040556 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:25.041407 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:25.041408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:25.042225 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:25.042227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:25.042716 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:25.042717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:25.042844 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:25.043436 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:30:25.043436 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:25.043468 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:30:25.043491 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:25.043676 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:25.043678 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:25.043681 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:30:25.043728 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:30:25.043812 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":102,"threads_started":1}
I20260809 16:30:25.044718 14537 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34555
I20260809 16:30:25.044828 14537 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:25.045051 14537 heartbeater.cc:511] Master 127.30.23.190:34555 requested a full tablet report, sending...
I20260809 16:30:25.045549 13794 ts_manager.cc:194] Registered new tserver with Master: f9b74b6cab3c43c2bd3dd8d817d5fe5a (127.30.23.131:38139)
I20260809 16:30:25.045960 13794 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.23.131:40335
I20260809 16:30:25.051642 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:30:25.054092 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 13886
W20260809 16:30:25.054119 13856 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:25.053274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53738 (local address 127.30.23.190:34555)
0809 16:30:25.053371 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:25.053374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:25.053521 (+ 147us) server_negotiation.cc:407] Connection header received
0809 16:30:25.053547 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:25.053548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:25.053559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:25.053605 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:25.053910 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53738: BlockingRecv error: recv got EOF from 127.0.0.1:53738 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:30:25.067957 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 14115
I20260809 16:30:25.089562 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 14327
I20260809 16:30:25.107278 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 13743
2026-08-09T16:30:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (888 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T16:30:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:30:25Z Disabled control of system clock
I20260809 16:30:25.143095 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:40433
--webserver_interface=127.30.23.190
--webserver_port=0
--builtin_ntp_servers=127.30.23.148:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:40433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:30:25.230482 14561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:25.230712 14561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:25.230736 14561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:25.231913 14561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:30:25.231954 14561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:25.231967 14561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:25.231980 14561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:30:25.231993 14561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:30:25.233331 14561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:40433
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:40433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:25.233656 14561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:25.233911 14561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:25.239300 14572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:25.239290 14577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:25.239329 14561 server_base.cc:1034] running on GCE node
W20260809 16:30:25.239290 14573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:25.240155 14561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:25.240487 14561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:25.241671 14561 hybrid_clock.cc:648] HybridClock initialized: now 1786293025241651 us; error 44 us; skew 500 ppm
I20260809 16:30:25.243280 14561 webserver.cc:460] Webserver started at http://127.30.23.190:43817/ using document root <none> and password file <none>
I20260809 16:30:25.243686 14561 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:25.243780 14561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:25.243979 14561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:25.245218 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "270272ee6679491889edd057687d3fde"
format_stamp: "Formatted at 2026-08-09 16:30:25 on dist-test-slave-j2vl"
server_key: "9c58024f9f88a22451b71d2eebee1015"
server_key_iv: "6eb009d5b93c395ca3ebf54a2666aaff"
server_key_version: "encryptionkey@0"
I20260809 16:30:25.245987 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "270272ee6679491889edd057687d3fde"
format_stamp: "Formatted at 2026-08-09 16:30:25 on dist-test-slave-j2vl"
server_key: "9c58024f9f88a22451b71d2eebee1015"
server_key_iv: "6eb009d5b93c395ca3ebf54a2666aaff"
server_key_version: "encryptionkey@0"
I20260809 16:30:25.247516 14561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:30:25.248492 14584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:25.248695 14561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:25.248826 14561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "270272ee6679491889edd057687d3fde"
format_stamp: "Formatted at 2026-08-09 16:30:25 on dist-test-slave-j2vl"
server_key: "9c58024f9f88a22451b71d2eebee1015"
server_key_iv: "6eb009d5b93c395ca3ebf54a2666aaff"
server_key_version: "encryptionkey@0"
I20260809 16:30:25.248916 14561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:25.253453 14561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:25.253911 14561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:25.254086 14561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:25.262822 14561 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:40433
I20260809 16:30:25.262933 14676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:40433 every 8 connection(s)
I20260809 16:30:25.263451 14561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:30:25.264303 14678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:25.268184 14678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde: Bootstrap starting.
I20260809 16:30:25.268603 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 14561
I20260809 16:30:25.268693 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:30:25.269038 30814 external_mini_cluster.cc:1428] Setting key b6722865b5a2880e7b9d3704c1c43a3f
I20260809 16:30:25.269029 14678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:25.269434 14678 log.cc:824] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde: Log is configured to *not* fsync() on all Append() calls
I20260809 16:30:25.270390 14678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde: No bootstrap required, opened a new log
I20260809 16:30:25.271806 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:25.269759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54178 (local address 127.30.23.190:40433)
0809 16:30:25.269958 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:25.269965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:25.269991 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:30:25.270457 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:25.270467 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:25.270523 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:25.270655 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:25.271334 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:25.271337 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:25.271337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:30:25.271341 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:25.271380 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:30:25.271390 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:25.271480 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:25.271530 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 16:30:25.271601 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
I20260809 16:30:25.272503 14678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270272ee6679491889edd057687d3fde" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40433 } }
I20260809 16:30:25.272679 14678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:25.272727 14678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270272ee6679491889edd057687d3fde, State: Initialized, Role: FOLLOWER
I20260809 16:30:25.272853 14678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270272ee6679491889edd057687d3fde" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40433 } }
I20260809 16:30:25.272938 14678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:30:25.272984 14678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:30:25.273032 14678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:25.274060 14678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270272ee6679491889edd057687d3fde" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40433 } }
I20260809 16:30:25.274173 14678 leader_election.cc:302] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 270272ee6679491889edd057687d3fde; no voters:
I20260809 16:30:25.274353 14678 leader_election.cc:288] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:30:25.274466 14685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:30:25.274633 14685 raft_consensus.cc:686] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [term 1 LEADER]: Becoming Leader. State: Replica: 270272ee6679491889edd057687d3fde, State: Running, Role: LEADER
I20260809 16:30:25.274638 14678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:30:25.274845 14685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270272ee6679491889edd057687d3fde" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40433 } }
I20260809 16:30:25.275312 14687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [sys.catalog]: SysCatalogTable state changed. Reason: New leader 270272ee6679491889edd057687d3fde. Latest consensus state: current_term: 1 leader_uuid: "270272ee6679491889edd057687d3fde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270272ee6679491889edd057687d3fde" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40433 } } }
I20260809 16:30:25.275810 14687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [sys.catalog]: This master's current role is: LEADER
I20260809 16:30:25.275710 14686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "270272ee6679491889edd057687d3fde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270272ee6679491889edd057687d3fde" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40433 } } }
I20260809 16:30:25.276047 14686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde [sys.catalog]: This master's current role is: LEADER
I20260809 16:30:25.276124 14698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:30:25.276733 14698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:30:25.278482 14698 catalog_manager.cc:1347] Generated new cluster ID: fcc9f17799604f43a8f622872cbe5ccf
I20260809 16:30:25.278548 14698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:30:25.300028 14698 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:30:25.300292 14698 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:30:25.308288 14698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 270272ee6679491889edd057687d3fde: Generated new TSK 0
I20260809 16:30:25.308512 14698 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:30:25.320151 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:40433
--builtin_ntp_servers=127.30.23.148:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:30:25.411309 14709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:25.411538 14709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:25.411564 14709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:25.412853 14709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:25.412902 14709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:25.412931 14709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:30:25.414464 14709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:40433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:25.414891 14709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:25.415380 14709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:30:25.422575 14709 server_base.cc:1034] running on GCE node
W20260809 16:30:25.422483 14717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:25.422480 14718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:25.422480 14720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:25.423105 14709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:25.423414 14709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:25.424620 14709 hybrid_clock.cc:648] HybridClock initialized: now 1786293025424604 us; error 39 us; skew 500 ppm
I20260809 16:30:25.426077 14709 webserver.cc:460] Webserver started at http://127.30.23.129:43835/ using document root <none> and password file <none>
I20260809 16:30:25.426373 14709 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:25.426456 14709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:25.426615 14709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:25.427829 14709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "3adda0a837364d548476cc9f87a8da44"
format_stamp: "Formatted at 2026-08-09 16:30:25 on dist-test-slave-j2vl"
server_key: "01ec04633e6c9eedcf1ffaaa854a624b"
server_key_iv: "309b9a7b81b07a18cb5b5fe778d60354"
server_key_version: "encryptionkey@0"
I20260809 16:30:25.428325 14709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "3adda0a837364d548476cc9f87a8da44"
format_stamp: "Formatted at 2026-08-09 16:30:25 on dist-test-slave-j2vl"
server_key: "01ec04633e6c9eedcf1ffaaa854a624b"
server_key_iv: "309b9a7b81b07a18cb5b5fe778d60354"
server_key_version: "encryptionkey@0"
I20260809 16:30:25.429939 14709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:30:25.430894 14731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:25.431151 14709 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:25.431354 14709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "3adda0a837364d548476cc9f87a8da44"
format_stamp: "Formatted at 2026-08-09 16:30:25 on dist-test-slave-j2vl"
server_key: "01ec04633e6c9eedcf1ffaaa854a624b"
server_key_iv: "309b9a7b81b07a18cb5b5fe778d60354"
server_key_version: "encryptionkey@0"
I20260809 16:30:25.431481 14709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:25.435191 14709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:25.435504 14709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:25.435658 14709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:25.435886 14709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:25.436385 14709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:25.436477 14709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:25.436525 14709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:25.436554 14709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:25.443028 14709 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:43415
I20260809 16:30:25.443231 14933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:43415 every 8 connection(s)
I20260809 16:30:25.443629 14709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:30:25.446507 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 14709
I20260809 16:30:25.446606 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:30:25.446874 30814 external_mini_cluster.cc:1428] Setting key 2bc62e491446b4c7e535d080af604861
I20260809 16:30:25.447218 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:25.444556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:34805 (local address 127.30.23.190:40433)
0809 16:30:25.444645 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:25.444647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:25.444850 (+ 203us) server_negotiation.cc:407] Connection header received
0809 16:30:25.445441 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:25.445443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:25.445451 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:25.445531 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:25.446899 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:25.446901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:25.446901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:30:25.446907 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:25.446923 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:30:25.446928 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:25.446985 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:25.447124 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 16:30:25.447143 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 16:30:25.447480 14938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:25.444521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40433 (local address 127.30.23.129:34805)
0809 16:30:25.444725 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:25.444763 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:25.445244 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:25.445600 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:25.445604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:25.445613 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:25.446704 (+ 1091us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:30:25.446704 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:25.446737 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:30:25.446770 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:25.447034 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:25.447040 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:25.447043 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:30:25.447137 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 16:30:25.447209 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
I20260809 16:30:25.448150 14935 heartbeater.cc:347] Connected to a master server at 127.30.23.190:40433
I20260809 16:30:25.448175 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:40433
--builtin_ntp_servers=127.30.23.148:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 16:30:25.448259 14935 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:25.448462 14935 heartbeater.cc:511] Master 127.30.23.190:40433 requested a full tablet report, sending...
I20260809 16:30:25.449143 14615 ts_manager.cc:194] Registered new tserver with Master: 3adda0a837364d548476cc9f87a8da44 (127.30.23.129:43415)
W20260809 16:30:25.536635 14939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:25.536901 14939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:25.536943 14939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:25.538233 14939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:25.538304 14939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:25.538352 14939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:30:25.539811 14939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:40433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:25.540112 14939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:25.540441 14939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:25.546216 14948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:25.546218 14947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:25.546216 14950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:25.546236 14939 server_base.cc:1034] running on GCE node
I20260809 16:30:25.546787 14939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:25.547113 14939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:25.548301 14939 hybrid_clock.cc:648] HybridClock initialized: now 1786293025548288 us; error 57 us; skew 500 ppm
I20260809 16:30:25.550226 14939 webserver.cc:460] Webserver started at http://127.30.23.130:32959/ using document root <none> and password file <none>
I20260809 16:30:25.550565 14939 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:25.550619 14939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:25.550765 14939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:25.551966 14939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "fb13a4eadbab42888613e4aff6cdc740"
format_stamp: "Formatted at 2026-08-09 16:30:25 on dist-test-slave-j2vl"
server_key: "420f1537afd2b47608e3a9efd02ba446"
server_key_iv: "27c8470280ca7739e88f39ac4279e966"
server_key_version: "encryptionkey@0"
I20260809 16:30:25.552387 14939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "fb13a4eadbab42888613e4aff6cdc740"
format_stamp: "Formatted at 2026-08-09 16:30:25 on dist-test-slave-j2vl"
server_key: "420f1537afd2b47608e3a9efd02ba446"
server_key_iv: "27c8470280ca7739e88f39ac4279e966"
server_key_version: "encryptionkey@0"
I20260809 16:30:25.553972 14939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:30:25.554855 14964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:25.555076 14939 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:25.555207 14939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "fb13a4eadbab42888613e4aff6cdc740"
format_stamp: "Formatted at 2026-08-09 16:30:25 on dist-test-slave-j2vl"
server_key: "420f1537afd2b47608e3a9efd02ba446"
server_key_iv: "27c8470280ca7739e88f39ac4279e966"
server_key_version: "encryptionkey@0"
I20260809 16:30:25.555317 14939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:25.559474 14939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:25.559800 14939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:25.559964 14939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:25.560200 14939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:25.560561 14939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:25.560621 14939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:25.560669 14939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:25.560696 14939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:25.567227 14939 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:35055
I20260809 16:30:25.567298 15164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:35055 every 8 connection(s)
I20260809 16:30:25.567699 14939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:30:25.571249 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:25.568612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:51389 (local address 127.30.23.190:40433)
0809 16:30:25.568694 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:25.568697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:25.569000 (+ 303us) server_negotiation.cc:407] Connection header received
0809 16:30:25.569614 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:25.569615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:25.569622 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:25.569687 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:25.570994 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:25.570996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:25.570996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:30:25.571001 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:25.571027 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:30:25.571032 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:25.571089 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:25.571176 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 16:30:25.571190 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:30:25.571480 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:25.568598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40433 (local address 127.30.23.130:51389)
0809 16:30:25.568856 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:25.568896 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:25.569504 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:25.569816 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:25.569820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:25.569829 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:25.570812 (+ 983us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:30:25.570813 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:25.570850 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:30:25.570882 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:25.571100 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:25.571106 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:25.571108 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:30:25.571172 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 16:30:25.571227 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":130,"threads_started":1}
I20260809 16:30:25.572005 15165 heartbeater.cc:347] Connected to a master server at 127.30.23.190:40433
I20260809 16:30:25.572103 15165 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:25.572232 15165 heartbeater.cc:511] Master 127.30.23.190:40433 requested a full tablet report, sending...
I20260809 16:30:25.572638 14615 ts_manager.cc:194] Registered new tserver with Master: fb13a4eadbab42888613e4aff6cdc740 (127.30.23.130:35055)
I20260809 16:30:25.575155 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 14939
I20260809 16:30:25.575313 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:30:25.575537 30814 external_mini_cluster.cc:1428] Setting key 68253f1d85f89e5c22c983c5fa018e6c
I20260809 16:30:25.576911 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:40433
--builtin_ntp_servers=127.30.23.148:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:30:25.661741 15169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:25.661971 15169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:25.661996 15169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:25.663214 15169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:25.663266 15169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:25.663295 15169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:30:25.664671 15169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:40433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:25.664954 15169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:25.665274 15169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:25.670625 15180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:25.670625 15175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:25.670734 15174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:25.670997 15169 server_base.cc:1034] running on GCE node
I20260809 16:30:25.671218 15169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:25.671536 15169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:25.672772 15169 hybrid_clock.cc:648] HybridClock initialized: now 1786293025672715 us; error 68 us; skew 500 ppm
I20260809 16:30:25.674319 15169 webserver.cc:460] Webserver started at http://127.30.23.131:43241/ using document root <none> and password file <none>
I20260809 16:30:25.674641 15169 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:25.674695 15169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:25.674907 15169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:25.676210 15169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "a7a9675cf1734f32b6278b5a106683e2"
format_stamp: "Formatted at 2026-08-09 16:30:25 on dist-test-slave-j2vl"
server_key: "088d1de48d0d06f00e22ba9199b0dbd6"
server_key_iv: "ca394f951c6682a836cdd0510207dabb"
server_key_version: "encryptionkey@0"
I20260809 16:30:25.676663 15169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "a7a9675cf1734f32b6278b5a106683e2"
format_stamp: "Formatted at 2026-08-09 16:30:25 on dist-test-slave-j2vl"
server_key: "088d1de48d0d06f00e22ba9199b0dbd6"
server_key_iv: "ca394f951c6682a836cdd0510207dabb"
server_key_version: "encryptionkey@0"
I20260809 16:30:25.678253 15169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:30:25.679271 15188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:25.679514 15169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:25.679626 15169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "a7a9675cf1734f32b6278b5a106683e2"
format_stamp: "Formatted at 2026-08-09 16:30:25 on dist-test-slave-j2vl"
server_key: "088d1de48d0d06f00e22ba9199b0dbd6"
server_key_iv: "ca394f951c6682a836cdd0510207dabb"
server_key_version: "encryptionkey@0"
I20260809 16:30:25.679713 15169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:25.683419 15169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:25.683710 15169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:25.683858 15169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:25.684091 15169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:25.684410 15169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:25.684446 15169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:25.684468 15169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:25.684480 15169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:25.690744 15169 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:45343
I20260809 16:30:25.690788 15384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:45343 every 8 connection(s)
I20260809 16:30:25.691164 15169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:30:25.692845 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 15169
I20260809 16:30:25.693005 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:30:25.693238 30814 external_mini_cluster.cc:1428] Setting key 22a737cea7272cda240890bbb39af1fc
I20260809 16:30:25.695032 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:25.692112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:60545 (local address 127.30.23.190:40433)
0809 16:30:25.692189 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:25.692191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:25.692419 (+ 228us) server_negotiation.cc:407] Connection header received
0809 16:30:25.692975 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:25.692976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:25.692984 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:25.693039 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:25.694732 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:25.694734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:25.694734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:30:25.694739 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:25.694757 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:30:25.694761 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:25.694828 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:25.694941 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 16:30:25.694958 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:30:25.695255 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:25.692035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:40433 (local address 127.30.23.131:60545)
0809 16:30:25.692249 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:25.692296 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:25.692848 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:25.693115 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:25.693120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:25.693138 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:25.694542 (+ 1404us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:30:25.694543 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:25.694576 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:30:25.694609 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:25.694841 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:25.694847 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:25.694850 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:30:25.694955 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 16:30:25.695026 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
I20260809 16:30:25.695690 15385 heartbeater.cc:347] Connected to a master server at 127.30.23.190:40433
I20260809 16:30:25.695791 15385 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:25.695912 15385 heartbeater.cc:511] Master 127.30.23.190:40433 requested a full tablet report, sending...
I20260809 16:30:25.696282 14615 ts_manager.cc:194] Registered new tserver with Master: a7a9675cf1734f32b6278b5a106683e2 (127.30.23.131:45343)
I20260809 16:30:25.705768 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 16:30:25.707883 14682 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:30:25.707353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54188 (local address 127.30.23.190:40433)
0809 16:30:25.707434 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:25.707436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:25.707505 (+ 69us) server_negotiation.cc:407] Connection header received
0809 16:30:25.707567 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:25.707567 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:25.707573 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:25.707631 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:25.707764 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54188: BlockingRecv error: recv got EOF from 127.0.0.1:54188 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:30:25.707923 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 14709
I20260809 16:30:25.723798 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 14939
I20260809 16:30:25.739465 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 15169
I20260809 16:30:25.747439 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 14561
2026-08-09T16:30:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (647 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:30:25 dist-test-slave-j2vl krb5kdc[15401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:30:25 dist-test-slave-j2vl krb5kdc[15401](info): set up 2 sockets
Aug 09 16:30:25 dist-test-slave-j2vl krb5kdc[15401](info): commencing operation
krb5kdc: starting...
W20260809 16:30:27.097784 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.317s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:30:27 dist-test-slave-j2vl krb5kdc[15401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:30:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:30:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:30:27.238055 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:45089
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:45381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:45089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:27.334100 15427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:27.334329 15427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:27.334354 15427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:27.335650 15427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:30:27.335700 15427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:27.335714 15427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:27.335755 15427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:30:27.335785 15427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:30:27.337594 15427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:45089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:45089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:27.337891 15427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:27.338189 15427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:27.343762 15446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:27.343773 15441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:27.343778 15440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:27.343955 15427 server_base.cc:1034] running on GCE node
I20260809 16:30:27.344336 15427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:27.344731 15427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:27.345903 15427 hybrid_clock.cc:648] HybridClock initialized: now 1786293027345885 us; error 41 us; skew 500 ppm
Aug 09 16:30:27 dist-test-slave-j2vl krb5kdc[15401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:27.348793 15427 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:30:27.349330 15427 webserver.cc:460] Webserver started at http://127.30.23.190:37023/ using document root <none> and password file <none>
I20260809 16:30:27.349637 15427 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:27.349712 15427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:27.349876 15427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:27.350975 15427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "b50d089215094426ae3c6b8a8637e51c"
format_stamp: "Formatted at 2026-08-09 16:30:27 on dist-test-slave-j2vl"
server_key: "6278abab985d733e5bbbd43fbd93ef99"
server_key_iv: "5e941978cfba902490662f203ab94b45"
server_key_version: "encryptionkey@0"
I20260809 16:30:27.351349 15427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "b50d089215094426ae3c6b8a8637e51c"
format_stamp: "Formatted at 2026-08-09 16:30:27 on dist-test-slave-j2vl"
server_key: "6278abab985d733e5bbbd43fbd93ef99"
server_key_iv: "5e941978cfba902490662f203ab94b45"
server_key_version: "encryptionkey@0"
I20260809 16:30:27.352733 15427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:30:27.353586 15455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:27.353798 15427 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:30:27.353919 15427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "b50d089215094426ae3c6b8a8637e51c"
format_stamp: "Formatted at 2026-08-09 16:30:27 on dist-test-slave-j2vl"
server_key: "6278abab985d733e5bbbd43fbd93ef99"
server_key_iv: "5e941978cfba902490662f203ab94b45"
server_key_version: "encryptionkey@0"
I20260809 16:30:27.353998 15427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:27.368336 15427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:27.371059 15427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:27.371254 15427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:27.376096 15427 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:45089
I20260809 16:30:27.376152 15546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:45089 every 8 connection(s)
I20260809 16:30:27.376545 15427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:30:27.377774 15547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:27.380136 15547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c: Bootstrap starting.
I20260809 16:30:27.380724 15547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:27.381062 15547 log.cc:824] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:30:27.382009 15547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c: No bootstrap required, opened a new log
I20260809 16:30:27.383687 15547 raft_consensus.cc:348] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50d089215094426ae3c6b8a8637e51c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45089 } }
I20260809 16:30:27.383837 15547 raft_consensus.cc:374] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:27.383860 15547 raft_consensus.cc:729] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50d089215094426ae3c6b8a8637e51c, State: Initialized, Role: FOLLOWER
I20260809 16:30:27.384042 15547 consensus_queue.cc:260] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50d089215094426ae3c6b8a8637e51c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45089 } }
I20260809 16:30:27.384122 15547 raft_consensus.cc:388] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:30:27.384177 15547 raft_consensus.cc:482] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:30:27.384227 15547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:27.384910 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 15427
I20260809 16:30:27.385039 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:30:27.385205 15547 raft_consensus.cc:504] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50d089215094426ae3c6b8a8637e51c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45089 } }
I20260809 16:30:27.385252 30814 external_mini_cluster.cc:1428] Setting key 48528181b27759147191fe1597b9c5b3
I20260809 16:30:27.385327 15547 leader_election.cc:302] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b50d089215094426ae3c6b8a8637e51c; no voters:
I20260809 16:30:27.385552 15547 leader_election.cc:288] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:30:27.385619 15553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:30:27.385835 15553 raft_consensus.cc:686] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [term 1 LEADER]: Becoming Leader. State: Replica: b50d089215094426ae3c6b8a8637e51c, State: Running, Role: LEADER
I20260809 16:30:27.386022 15553 consensus_queue.cc:237] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50d089215094426ae3c6b8a8637e51c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45089 } }
I20260809 16:30:27.385864 15547 sys_catalog.cc:564] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:30:27.386469 15555 sys_catalog.cc:455] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b50d089215094426ae3c6b8a8637e51c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50d089215094426ae3c6b8a8637e51c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45089 } } }
I20260809 16:30:27.386550 15555 sys_catalog.cc:458] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [sys.catalog]: This master's current role is: LEADER
I20260809 16:30:27.386804 15557 sys_catalog.cc:455] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b50d089215094426ae3c6b8a8637e51c. Latest consensus state: current_term: 1 leader_uuid: "b50d089215094426ae3c6b8a8637e51c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50d089215094426ae3c6b8a8637e51c" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45089 } } }
I20260809 16:30:27.386929 15557 sys_catalog.cc:458] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c [sys.catalog]: This master's current role is: LEADER
W20260809 16:30:27.388338 15577 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:30:27.388407 15577 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:30:27.388461 15562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:30:27.388669 15562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:30:27.390990 15562 catalog_manager.cc:1347] Generated new cluster ID: c77be475e0bf45c7863d73160787d288
I20260809 16:30:27.391067 15562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:30:27 dist-test-slave-j2vl krb5kdc[15401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:30:27.396903 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.385974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.30.23.190:45089)
0809 16:30:27.386239 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:27.386248 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:27.386275 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:30:27.386404 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:27.386412 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:27.386476 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:27.387558 (+ 1082us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:27.387681 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.388969 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.389914 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.390345 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.390495 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.393391 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:27.393415 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:27.393415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:27.393431 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:27.395173 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:27.395617 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:27.395624 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:27.395624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:27.395730 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:27.395952 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:27.395953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:27.395954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:27.396232 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:27.396382 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:27.396513 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 16:30:27.396674 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":59,"threads_started":1}
I20260809 16:30:27.405001 15562 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:30:27.405701 15562 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:30:27.409936 15562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c: Generated new TSK 0
I20260809 16:30:27.410140 15562 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:30:27.441653 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45089
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:45381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:27.528393 15584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:27.528649 15584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:27.528693 15584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:27.529935 15584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:27.530009 15584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:27.530055 15584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:30:27.531591 15584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:27.531876 15584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:27.532122 15584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:27.537549 15603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:27.537549 15599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:27.537541 15598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:27.537567 15584 server_base.cc:1034] running on GCE node
I20260809 16:30:27.538105 15584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:27.538532 15584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:27.539777 15584 hybrid_clock.cc:648] HybridClock initialized: now 1786293027539737 us; error 54 us; skew 500 ppm
Aug 09 16:30:27 dist-test-slave-j2vl krb5kdc[15401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:27.542742 15584 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:30:27.543309 15584 webserver.cc:460] Webserver started at http://127.30.23.129:42363/ using document root <none> and password file <none>
I20260809 16:30:27.543603 15584 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:27.543684 15584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:27.543861 15584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:27.545081 15584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "8f298628a19942269b24ed5dcb9f2f7a"
format_stamp: "Formatted at 2026-08-09 16:30:27 on dist-test-slave-j2vl"
server_key: "fc61c8f6306be94f441e2c16d9c2ec75"
server_key_iv: "a09b1e44e4dd11cecd8e02a6f71c3f02"
server_key_version: "encryptionkey@0"
I20260809 16:30:27.545565 15584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "8f298628a19942269b24ed5dcb9f2f7a"
format_stamp: "Formatted at 2026-08-09 16:30:27 on dist-test-slave-j2vl"
server_key: "fc61c8f6306be94f441e2c16d9c2ec75"
server_key_iv: "a09b1e44e4dd11cecd8e02a6f71c3f02"
server_key_version: "encryptionkey@0"
I20260809 16:30:27.547466 15584 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260809 16:30:27.548465 15614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:27.548731 15584 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:27.548862 15584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "8f298628a19942269b24ed5dcb9f2f7a"
format_stamp: "Formatted at 2026-08-09 16:30:27 on dist-test-slave-j2vl"
server_key: "fc61c8f6306be94f441e2c16d9c2ec75"
server_key_iv: "a09b1e44e4dd11cecd8e02a6f71c3f02"
server_key_version: "encryptionkey@0"
I20260809 16:30:27.548960 15584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:27.563310 15584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:27.565790 15584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:27.565991 15584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:27.566277 15584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:27.566843 15584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:27.566912 15584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:27.566962 15584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:27.567010 15584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:27.573422 15584 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:43993
I20260809 16:30:27.573462 15810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:43993 every 8 connection(s)
I20260809 16:30:27.573827 15584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:30:27.578598 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 15584
I20260809 16:30:27.578732 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:30:27.578931 30814 external_mini_cluster.cc:1428] Setting key d64be2dc1a41c3656e34063cf3e8c65f
Aug 09 16:30:27 dist-test-slave-j2vl krb5kdc[15401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:30:27.584096 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.574723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:38839 (local address 127.30.23.190:45089)
0809 16:30:27.574807 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:27.574809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:27.575072 (+ 263us) server_negotiation.cc:407] Connection header received
0809 16:30:27.575592 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:27.575593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:27.575609 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:27.575669 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:27.576639 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.577159 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.578297 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.578539 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.578641 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.581042 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:27.581044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:27.581044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:27.581053 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:27.582571 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:27.583030 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:27.583031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:27.583032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:27.583101 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:27.583349 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:27.583350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:27.583350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:27.583506 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:27.583578 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:27.583830 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 16:30:27.583951 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 16:30:27.584113 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.574677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:45089 (local address 127.30.23.129:38839)
0809 16:30:27.574925 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:27.574964 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:27.575472 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:27.575717 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:27.575721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:27.576338 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:27.576541 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.576545 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.577266 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.577267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.578208 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.578211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.578654 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.578655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.578840 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.579429 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:27.579437 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:27.580939 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:27.582653 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:27.582659 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:27.582661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:27.582941 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:27.583175 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:27.583175 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:27.583175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:27.583278 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:27.583571 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:27.583576 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:27.583697 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 16:30:27.583783 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:30:27.583943 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":144,"threads_started":1}
I20260809 16:30:27.584730 15811 heartbeater.cc:347] Connected to a master server at 127.30.23.190:45089
I20260809 16:30:27.584889 15811 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:27.585095 15811 heartbeater.cc:511] Master 127.30.23.190:45089 requested a full tablet report, sending...
I20260809 16:30:27.585806 15485 ts_manager.cc:194] Registered new tserver with Master: 8f298628a19942269b24ed5dcb9f2f7a (127.30.23.129:43993)
I20260809 16:30:27.586484 15485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:38839
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:30:27.613154 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45089
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:45381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:27.698315 15817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:27.698536 15817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:27.698562 15817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:27.699787 15817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:27.699836 15817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:27.699864 15817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:30:27.701377 15817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:27.701632 15817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:27.701913 15817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:27.707134 15830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:27.707137 15828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:27.707170 15833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:27.707464 15817 server_base.cc:1034] running on GCE node
I20260809 16:30:27.707695 15817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:27.707970 15817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:27.709189 15817 hybrid_clock.cc:648] HybridClock initialized: now 1786293027709172 us; error 36 us; skew 500 ppm
Aug 09 16:30:27 dist-test-slave-j2vl krb5kdc[15401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:27.711948 15817 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:30:27.712440 15817 webserver.cc:460] Webserver started at http://127.30.23.130:35877/ using document root <none> and password file <none>
I20260809 16:30:27.712682 15817 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:27.712755 15817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:27.712909 15817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:27.714072 15817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "aeb2d3dfb23345e08d3af06cff7d5176"
format_stamp: "Formatted at 2026-08-09 16:30:27 on dist-test-slave-j2vl"
server_key: "84529a0edc6176e638e8bf53c879ce0e"
server_key_iv: "7daae702989b63ced1fb14564c9eb8b3"
server_key_version: "encryptionkey@0"
I20260809 16:30:27.714454 15817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "aeb2d3dfb23345e08d3af06cff7d5176"
format_stamp: "Formatted at 2026-08-09 16:30:27 on dist-test-slave-j2vl"
server_key: "84529a0edc6176e638e8bf53c879ce0e"
server_key_iv: "7daae702989b63ced1fb14564c9eb8b3"
server_key_version: "encryptionkey@0"
I20260809 16:30:27.715873 15817 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:30:27.716733 15842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:27.717074 15817 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:27.717164 15817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "aeb2d3dfb23345e08d3af06cff7d5176"
format_stamp: "Formatted at 2026-08-09 16:30:27 on dist-test-slave-j2vl"
server_key: "84529a0edc6176e638e8bf53c879ce0e"
server_key_iv: "7daae702989b63ced1fb14564c9eb8b3"
server_key_version: "encryptionkey@0"
I20260809 16:30:27.717219 15817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:27.729545 15817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:27.731902 15817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:27.732026 15817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:27.732223 15817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:27.732662 15817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:27.732699 15817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:27.732722 15817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:27.732735 15817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:27.738816 15817 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:43905
I20260809 16:30:27.738870 16022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:43905 every 8 connection(s)
I20260809 16:30:27.739228 15817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:30:27.739730 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 15817
I20260809 16:30:27.739871 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:30:27.740127 30814 external_mini_cluster.cc:1428] Setting key ae78b024f64b5ccc12c29579e253e424
Aug 09 16:30:27 dist-test-slave-j2vl krb5kdc[15401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
I20260809 16:30:27.750494 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.740102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:34897 (local address 127.30.23.190:45089)
0809 16:30:27.740175 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:27.740178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:27.740458 (+ 280us) server_negotiation.cc:407] Connection header received
0809 16:30:27.741092 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:27.741093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:27.741104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:27.741144 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:27.742543 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.743110 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.744207 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.744413 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.744498 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.747128 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:27.747131 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:27.747131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:27.747140 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:27.748902 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:27.749327 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:27.749328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:27.749328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:27.749415 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:27.749723 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:27.749725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:27.749725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:27.749872 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:27.749946 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:27.750301 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 16:30:27.750393 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:30:27.750618 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.740085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:45089 (local address 127.30.23.130:34897)
0809 16:30:27.740286 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:27.740346 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:27.740987 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:27.741216 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:27.741220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:27.742134 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:27.742429 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.742435 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.743222 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.743224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.744110 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.744112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.744504 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.744506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.744637 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.745223 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:27.745235 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:27.747020 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:27.748994 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:27.749001 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:27.749003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:27.749234 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:27.749513 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:27.749514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:27.749514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:27.749639 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:27.749975 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:27.749982 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:27.750167 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 16:30:27.750258 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 16:30:27.750402 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":100,"threads_started":1}
I20260809 16:30:27.751260 16024 heartbeater.cc:347] Connected to a master server at 127.30.23.190:45089
I20260809 16:30:27.751374 16024 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:27.751542 16024 heartbeater.cc:511] Master 127.30.23.190:45089 requested a full tablet report, sending...
I20260809 16:30:27.752038 15485 ts_manager.cc:194] Registered new tserver with Master: aeb2d3dfb23345e08d3af06cff7d5176 (127.30.23.130:43905)
I20260809 16:30:27.752602 15485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:34897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:30:27.774001 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45089
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:45381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:30:27.864287 16032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:30:27.864574 16032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:30:27.864624 16032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:30:27.865928 16032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:30:27.866003 16032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:30:27.866050 16032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:30:27.867774 16032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:30:27.868108 16032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:30:27.868369 16032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:30:27.874046 16043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:27.874060 16039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:30:27.874060 16038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:30:27.874543 16032 server_base.cc:1034] running on GCE node
I20260809 16:30:27.874778 16032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:30:27.875010 16032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:30:27.876178 16032 hybrid_clock.cc:648] HybridClock initialized: now 1786293027876158 us; error 32 us; skew 500 ppm
Aug 09 16:30:27 dist-test-slave-j2vl krb5kdc[15401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:30:27.879186 16032 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:30:27.879714 16032 webserver.cc:460] Webserver started at http://127.30.23.131:42055/ using document root <none> and password file <none>
I20260809 16:30:27.879957 16032 fs_manager.cc:359] Metadata directory not provided
I20260809 16:30:27.880030 16032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:30:27.880192 16032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:30:27.881294 16032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0"
format_stamp: "Formatted at 2026-08-09 16:30:27 on dist-test-slave-j2vl"
server_key: "9bd11843fdeffb5490d392834f12186e"
server_key_iv: "78596a472b60722e14a85ebad48eb291"
server_key_version: "encryptionkey@0"
I20260809 16:30:27.881942 16032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0"
format_stamp: "Formatted at 2026-08-09 16:30:27 on dist-test-slave-j2vl"
server_key: "9bd11843fdeffb5490d392834f12186e"
server_key_iv: "78596a472b60722e14a85ebad48eb291"
server_key_version: "encryptionkey@0"
I20260809 16:30:27.883467 16032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:30:27.884382 16053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:27.884706 16032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:30:27.884847 16032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0"
format_stamp: "Formatted at 2026-08-09 16:30:27 on dist-test-slave-j2vl"
server_key: "9bd11843fdeffb5490d392834f12186e"
server_key_iv: "78596a472b60722e14a85ebad48eb291"
server_key_version: "encryptionkey@0"
I20260809 16:30:27.884944 16032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:30:27.894394 16032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:30:27.897977 16032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:30:27.898180 16032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:30:27.898537 16032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:30:27.898926 16032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:30:27.899000 16032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:27.899048 16032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:30:27.899077 16032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:30:27.906632 16032 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:35737
I20260809 16:30:27.906672 16229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:35737 every 8 connection(s)
I20260809 16:30:27.907032 16032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:30:27.910600 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 16032
I20260809 16:30:27.910710 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:30:27.910902 30814 external_mini_cluster.cc:1428] Setting key b1fb3269d7c5d17ebaf9b8a965383244
Aug 09 16:30:27 dist-test-slave-j2vl krb5kdc[15401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293027, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:30:27.919396 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.907892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:49403 (local address 127.30.23.190:45089)
0809 16:30:27.907950 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:27.907952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:27.908217 (+ 265us) server_negotiation.cc:407] Connection header received
0809 16:30:27.909016 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:27.909017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:27.909029 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:27.909085 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:27.910517 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.911276 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.912878 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.913072 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.913128 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.916000 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:27.916002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:27.916002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:27.916011 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:27.917840 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:27.918278 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:27.918280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:27.918281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:27.918344 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:27.918637 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:27.918638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:27.918639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:27.918794 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:27.918875 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:27.919176 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 16:30:27.919270 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:30:27.919593 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.907843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:45089 (local address 127.30.23.131:49403)
0809 16:30:27.908049 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:27.908106 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:27.908868 (+ 762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:27.909168 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:27.909173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:27.910092 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:30:27.910421 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.910428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.911381 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.911382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.912792 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.912796 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.913161 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.913162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.913373 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.913980 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:30:27.913992 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:30:27.915895 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:30:27.917948 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:27.917955 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:27.917957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:27.918198 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:27.918438 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:30:27.918439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:30:27.918439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:30:27.918554 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:30:27.918893 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:30:27.918903 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:30:27.919033 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 16:30:27.919130 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 16:30:27.919265 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":101,"threads_started":1}
I20260809 16:30:27.920264 16230 heartbeater.cc:347] Connected to a master server at 127.30.23.190:45089
I20260809 16:30:27.920387 16230 heartbeater.cc:464] Registering TS with master...
I20260809 16:30:27.920598 16230 heartbeater.cc:511] Master 127.30.23.190:45089 requested a full tablet report, sending...
I20260809 16:30:27.921087 15485 ts_manager.cc:194] Registered new tserver with Master: a419bc3e42cb42b2a9b0201e1b5f72a0 (127.30.23.131:35737)
I20260809 16:30:27.921665 15485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:49403
I20260809 16:30:27.922534 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:30:27.931315 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.923873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.30.23.190:45089)
0809 16:30:27.923914 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:27.923916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:27.924001 (+ 85us) server_negotiation.cc:407] Connection header received
0809 16:30:27.924043 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:27.924044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:27.924055 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:27.924128 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:30:27.924657 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.925586 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.926412 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.926942 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.927042 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.928352 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:30:27.928353 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:30:27.928354 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:30:27.928363 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:30:27.930019 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:27.930383 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:27.930385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:27.930385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:27.930453 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:30:27.930650 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:30:27.930651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:30:27.930651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:30:27.930873 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:30:27.930981 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:27.931082 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 16:30:27.931176 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:30:27.933305 15485 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:30:27.935170 15485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:30:27.949009 16247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.943797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:43905 (local address 127.0.0.1:51072)
0809 16:30:27.943981 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:27.944033 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:27.944112 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:27.944770 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:27.944771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:27.944773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:27.944820 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.944822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.946232 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.946233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.947683 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.947687 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.948669 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.948670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.948762 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.948779 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 16:30:27.948830 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:30:27.948892 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":45,"thread_start_us":59,"threads_started":1}
I20260809 16:30:27.949271 16252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.943801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:43993 (local address 127.0.0.1:57780)
0809 16:30:27.944248 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:27.944257 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:27.944304 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:27.944597 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:27.944600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:27.944603 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:27.944691 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.944693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.946226 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.946227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.947700 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.947701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.949035 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.949036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.949112 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.949118 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:30:27.949164 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:30:27.949206 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":268,"thread_start_us":138,"threads_started":1}
I20260809 16:30:27.949519 16250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.943927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51072 (local address 127.30.23.130:43905)
0809 16:30:27.944548 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:27.944553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:27.944563 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:30:27.944601 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:27.944602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:27.944617 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:27.944688 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:27.944895 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.946105 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.947793 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.948562 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.949245 (+ 683us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.949317 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:27.949385 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 16:30:27.949439 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":86,"threads_started":1}
I20260809 16:30:27.949646 16251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.944003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57780 (local address 127.30.23.129:43993)
0809 16:30:27.944227 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:27.944232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:27.944297 (+ 65us) server_negotiation.cc:407] Connection header received
0809 16:30:27.944373 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:27.944376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:27.944395 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:27.944486 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:27.944772 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.946101 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.947793 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.948562 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.948638 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.949477 (+ 839us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:27.949503 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 16:30:27.949554 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":104,"threads_started":1}
I20260809 16:30:27.950314 16249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.944064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:35737 (local address 127.0.0.1:34982)
0809 16:30:27.944382 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:27.944391 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:27.944425 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:27.945437 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:27.945439 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:27.945442 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:27.945505 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.945507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.947052 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.947053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.948327 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.948331 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.950070 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.950071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.950134 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.950139 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:30:27.950198 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:30:27.950240 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":73,"threads_started":1}
I20260809 16:30:27.950316 15927 tablet_service.cc:1467] Processing CreateTablet for tablet bfdc74c3e2b34695b47bb5950ad4beb7 (DEFAULT_TABLE table=test-table [id=695113444b2d47f99ed0b48bc389888a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:27.950629 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.944263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34982 (local address 127.30.23.131:35737)
0809 16:30:27.945129 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:27.945134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:27.945148 (+ 14us) server_negotiation.cc:407] Connection header received
0809 16:30:27.945202 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:27.945205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:27.945222 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:27.945313 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:27.945593 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.946931 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.948435 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.949969 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.950047 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.950097 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:27.950505 (+ 408us) server_negotiation.cc:299] Negotiation successful
0809 16:30:27.950555 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":820,"thread_start_us":282,"threads_started":1}
I20260809 16:30:27.950742 15927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfdc74c3e2b34695b47bb5950ad4beb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:27.951320 16130 tablet_service.cc:1467] Processing CreateTablet for tablet bfdc74c3e2b34695b47bb5950ad4beb7 (DEFAULT_TABLE table=test-table [id=695113444b2d47f99ed0b48bc389888a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:27.951705 16130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfdc74c3e2b34695b47bb5950ad4beb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:27.954452 16254 tablet_bootstrap.cc:492] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176: Bootstrap starting.
I20260809 16:30:27.954989 15697 tablet_service.cc:1467] Processing CreateTablet for tablet bfdc74c3e2b34695b47bb5950ad4beb7 (DEFAULT_TABLE table=test-table [id=695113444b2d47f99ed0b48bc389888a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:30:27.955238 16256 tablet_bootstrap.cc:492] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0: Bootstrap starting.
I20260809 16:30:27.955366 15697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfdc74c3e2b34695b47bb5950ad4beb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:30:27.955636 16254 tablet_bootstrap.cc:654] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:27.955927 16256 tablet_bootstrap.cc:654] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:27.956310 16254 log.cc:824] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176: Log is configured to *not* fsync() on all Append() calls
I20260809 16:30:27.956552 16256 log.cc:824] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0: Log is configured to *not* fsync() on all Append() calls
I20260809 16:30:27.957685 16254 tablet_bootstrap.cc:492] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176: No bootstrap required, opened a new log
I20260809 16:30:27.957767 16254 ts_tablet_manager.cc:1397] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:30:27.959067 16260 tablet_bootstrap.cc:492] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a: Bootstrap starting.
I20260809 16:30:27.959806 16260 tablet_bootstrap.cc:654] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a: Neither blocks nor log segments found. Creating new log.
I20260809 16:30:27.959908 16256 tablet_bootstrap.cc:492] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0: No bootstrap required, opened a new log
I20260809 16:30:27.959990 16256 ts_tablet_manager.cc:1397] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 16:30:27.960100 16254 raft_consensus.cc:348] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43905 } } peers { permanent_uuid: "8f298628a19942269b24ed5dcb9f2f7a" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 43993 } } peers { permanent_uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35737 } }
I20260809 16:30:27.960269 16254 raft_consensus.cc:374] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:27.960306 16254 raft_consensus.cc:729] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeb2d3dfb23345e08d3af06cff7d5176, State: Initialized, Role: FOLLOWER
I20260809 16:30:27.960422 16260 log.cc:824] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:30:27.960451 16254 consensus_queue.cc:260] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43905 } } peers { permanent_uuid: "8f298628a19942269b24ed5dcb9f2f7a" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 43993 } } peers { permanent_uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35737 } }
I20260809 16:30:27.960737 16254 ts_tablet_manager.cc:1428] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:30:27.960887 16024 heartbeater.cc:503] Master 127.30.23.190:45089 was elected leader, sending a full tablet report...
I20260809 16:30:27.961956 16256 raft_consensus.cc:348] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43905 } } peers { permanent_uuid: "8f298628a19942269b24ed5dcb9f2f7a" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 43993 } } peers { permanent_uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35737 } }
I20260809 16:30:27.962078 16256 raft_consensus.cc:374] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:27.962097 16256 raft_consensus.cc:729] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a419bc3e42cb42b2a9b0201e1b5f72a0, State: Initialized, Role: FOLLOWER
I20260809 16:30:27.962180 16256 consensus_queue.cc:260] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43905 } } peers { permanent_uuid: "8f298628a19942269b24ed5dcb9f2f7a" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 43993 } } peers { permanent_uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35737 } }
I20260809 16:30:27.962456 16256 ts_tablet_manager.cc:1428] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:30:27.962473 16230 heartbeater.cc:503] Master 127.30.23.190:45089 was elected leader, sending a full tablet report...
I20260809 16:30:27.963264 16260 tablet_bootstrap.cc:492] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a: No bootstrap required, opened a new log
I20260809 16:30:27.963335 16260 ts_tablet_manager.cc:1397] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 16:30:27.965816 16260 raft_consensus.cc:348] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43905 } } peers { permanent_uuid: "8f298628a19942269b24ed5dcb9f2f7a" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 43993 } } peers { permanent_uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35737 } }
I20260809 16:30:27.966198 16260 raft_consensus.cc:374] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:30:27.966250 16260 raft_consensus.cc:729] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f298628a19942269b24ed5dcb9f2f7a, State: Initialized, Role: FOLLOWER
I20260809 16:30:27.966373 16260 consensus_queue.cc:260] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43905 } } peers { permanent_uuid: "8f298628a19942269b24ed5dcb9f2f7a" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 43993 } } peers { permanent_uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35737 } }
I20260809 16:30:27.966711 16260 ts_tablet_manager.cc:1428] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 16:30:27.966893 15811 heartbeater.cc:503] Master 127.30.23.190:45089 was elected leader, sending a full tablet report...
W20260809 16:30:27.989876 16026 tablet.cc:2369] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:30:27.991842 16261 raft_consensus.cc:482] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:30:27.992020 16261 raft_consensus.cc:504] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43905 } } peers { permanent_uuid: "8f298628a19942269b24ed5dcb9f2f7a" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 43993 } } peers { permanent_uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35737 } }
I20260809 16:30:27.992530 16261 leader_election.cc:288] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f298628a19942269b24ed5dcb9f2f7a (127.30.23.129:43993), a419bc3e42cb42b2a9b0201e1b5f72a0 (127.30.23.131:35737)
I20260809 16:30:27.996251 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.992980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:46559 (local address 127.30.23.131:35737)
0809 16:30:27.993045 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:27.993048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:27.993149 (+ 101us) server_negotiation.cc:407] Connection header received
0809 16:30:27.993179 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:27.993179 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:27.993187 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:27.993278 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:27.993455 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.994124 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.995384 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.995865 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.995944 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.995954 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:27.996136 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 16:30:27.996178 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:30:27.996272 16265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.992854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:35737 (local address 127.30.23.130:46559)
0809 16:30:27.993059 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:27.993071 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:27.993134 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:27.993284 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:27.993285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:27.993286 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:27.993373 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.993375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.994219 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.994220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.995255 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.995259 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.995960 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.995961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.996044 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.996050 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:30:27.996111 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 16:30:27.996159 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":16,"thread_start_us":137,"threads_started":1}
I20260809 16:30:27.996719 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.992713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:43993 (local address 127.30.23.130:57921)
0809 16:30:27.992771 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:27.992785 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:30:27.992830 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:30:27.993085 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:30:27.993086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:30:27.993088 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:30:27.993163 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.993165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.994182 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.994184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.995626 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:30:27.995630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.996441 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.996442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:30:27.996533 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.996536 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:30:27.996608 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:30:27.996650 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":29}
I20260809 16:30:27.996719 16251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:27.992786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:57921 (local address 127.30.23.129:43993)
0809 16:30:27.992836 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:27.992838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:27.992878 (+ 40us) server_negotiation.cc:407] Connection header received
0809 16:30:27.992907 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:27.992908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:27.992920 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:27.992972 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:30:27.993256 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.994075 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.995748 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:27.996367 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:27.996442 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:27.996450 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:27.996616 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 16:30:27.996657 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:30:27.997112 15730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfdc74c3e2b34695b47bb5950ad4beb7" candidate_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f298628a19942269b24ed5dcb9f2f7a" is_pre_election: true
I20260809 16:30:27.997112 16164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfdc74c3e2b34695b47bb5950ad4beb7" candidate_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0" is_pre_election: true
I20260809 16:30:27.997292 15730 raft_consensus.cc:2454] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeb2d3dfb23345e08d3af06cff7d5176 in term 0.
I20260809 16:30:27.997305 16164 raft_consensus.cc:2454] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeb2d3dfb23345e08d3af06cff7d5176 in term 0.
I20260809 16:30:27.997604 15844 leader_election.cc:302] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a419bc3e42cb42b2a9b0201e1b5f72a0, aeb2d3dfb23345e08d3af06cff7d5176; no voters:
I20260809 16:30:27.997823 16261 raft_consensus.cc:2781] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:30:27.997882 16261 raft_consensus.cc:482] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:30:27.997911 16261 raft_consensus.cc:3037] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:27.998978 16261 raft_consensus.cc:504] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43905 } } peers { permanent_uuid: "8f298628a19942269b24ed5dcb9f2f7a" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 43993 } } peers { permanent_uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35737 } }
I20260809 16:30:27.999274 16261 leader_election.cc:288] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [CANDIDATE]: Term 1 election: Requested vote from peers 8f298628a19942269b24ed5dcb9f2f7a (127.30.23.129:43993), a419bc3e42cb42b2a9b0201e1b5f72a0 (127.30.23.131:35737)
I20260809 16:30:27.999372 16164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfdc74c3e2b34695b47bb5950ad4beb7" candidate_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0"
I20260809 16:30:27.999392 15730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfdc74c3e2b34695b47bb5950ad4beb7" candidate_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f298628a19942269b24ed5dcb9f2f7a"
I20260809 16:30:27.999464 16164 raft_consensus.cc:3037] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:27.999472 15730 raft_consensus.cc:3037] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:30:28.000458 16164 raft_consensus.cc:2454] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeb2d3dfb23345e08d3af06cff7d5176 in term 1.
I20260809 16:30:28.000461 15730 raft_consensus.cc:2454] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeb2d3dfb23345e08d3af06cff7d5176 in term 1.
I20260809 16:30:28.000622 15844 leader_election.cc:302] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a419bc3e42cb42b2a9b0201e1b5f72a0, aeb2d3dfb23345e08d3af06cff7d5176; no voters:
I20260809 16:30:28.000720 16261 raft_consensus.cc:2781] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:30:28.000942 16261 raft_consensus.cc:686] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [term 1 LEADER]: Becoming Leader. State: Replica: aeb2d3dfb23345e08d3af06cff7d5176, State: Running, Role: LEADER
I20260809 16:30:28.001055 16261 consensus_queue.cc:237] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43905 } } peers { permanent_uuid: "8f298628a19942269b24ed5dcb9f2f7a" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 43993 } } peers { permanent_uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35737 } }
I20260809 16:30:28.002068 15485 catalog_manager.cc:5495] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 reported cstate change: term changed from 0 to 1, leader changed from <none> to aeb2d3dfb23345e08d3af06cff7d5176 (127.30.23.130). New cstate: current_term: 1 leader_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb2d3dfb23345e08d3af06cff7d5176" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f298628a19942269b24ed5dcb9f2f7a" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35737 } health_report { overall_health: UNKNOWN } } }
I20260809 16:30:28.009907 16250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:28.006865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51076 (local address 127.30.23.130:43905)
0809 16:30:28.006917 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:30:28.006919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:30:28.006930 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:30:28.006993 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:30:28.006994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:30:28.007013 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:30:28.007072 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:30:28.007218 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:28.007995 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:28.008782 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:30:28.009129 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:30:28.009207 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:30:28.009469 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:30:28.009617 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:30:28.009709 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 16:30:28.009778 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 16:30:28.009804 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:30:28.011919 16164 raft_consensus.cc:1261] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0 [term 1 FOLLOWER]: Refusing update from remote peer aeb2d3dfb23345e08d3af06cff7d5176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:28.011919 15730 raft_consensus.cc:1261] T bfdc74c3e2b34695b47bb5950ad4beb7 P 8f298628a19942269b24ed5dcb9f2f7a [term 1 FOLLOWER]: Refusing update from remote peer aeb2d3dfb23345e08d3af06cff7d5176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:30:28.012317 16266 consensus_queue.cc:1035] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f298628a19942269b24ed5dcb9f2f7a" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:28.012455 16261 consensus_queue.cc:1035] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a419bc3e42cb42b2a9b0201e1b5f72a0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:30:28.015652 16274 mvcc.cc:205] Tried to move back new op lower bound from 7316656242733568000 to 7316656242693021696. Current Snapshot: MvccSnapshot[applied={T|T < 7316656242733568000}]
I20260809 16:30:28.015516 16269 mvcc.cc:205] Tried to move back new op lower bound from 7316656242733568000 to 7316656242693021696. Current Snapshot: MvccSnapshot[applied={T|T < 7316656242733568000}]
I20260809 16:30:28.015887 16271 mvcc.cc:205] Tried to move back new op lower bound from 7316656242733568000 to 7316656242693021696. Current Snapshot: MvccSnapshot[applied={T|T < 7316656242733568000}]
I20260809 16:30:28.018832 15484 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:30:28.018947 15484 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:30:28.019920 15484 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b50d089215094426ae3c6b8a8637e51c: Sending DeleteTablet for 3 replicas of tablet bfdc74c3e2b34695b47bb5950ad4beb7
I20260809 16:30:28.020372 15697 tablet_service.cc:1514] Processing DeleteTablet for tablet bfdc74c3e2b34695b47bb5950ad4beb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:28 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:57780
I20260809 16:30:28.020516 16130 tablet_service.cc:1514] Processing DeleteTablet for tablet bfdc74c3e2b34695b47bb5950ad4beb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:28 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:34982
I20260809 16:30:28.020615 15927 tablet_service.cc:1514] Processing DeleteTablet for tablet bfdc74c3e2b34695b47bb5950ad4beb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:30:28 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:51072
I20260809 16:30:28.020952 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 15584
I20260809 16:30:28.040206 16291 tablet_replica.cc:330] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0: stopping tablet replica
I20260809 16:30:28.040407 16291 raft_consensus.cc:2229] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:30:28.040555 16291 raft_consensus.cc:2258] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:28.040704 16293 tablet_replica.cc:330] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176: stopping tablet replica
I20260809 16:30:28.040910 16293 raft_consensus.cc:2229] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:30:28.041126 16293 raft_consensus.cc:2258] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:30:28.041134 16291 ts_tablet_manager.cc:1905] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:28.041795 16293 ts_tablet_manager.cc:1905] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:30:28.043810 16291 ts_tablet_manager.cc:1918] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:30:28.043905 16291 log.cc:1196] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/bfdc74c3e2b34695b47bb5950ad4beb7
I20260809 16:30:28.044076 16293 ts_tablet_manager.cc:1918] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:30:28.044211 16291 ts_tablet_manager.cc:1939] T bfdc74c3e2b34695b47bb5950ad4beb7 P a419bc3e42cb42b2a9b0201e1b5f72a0: Deleting consensus metadata
I20260809 16:30:28.044235 16293 log.cc:1196] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/bfdc74c3e2b34695b47bb5950ad4beb7
I20260809 16:30:28.044656 16293 ts_tablet_manager.cc:1939] T bfdc74c3e2b34695b47bb5950ad4beb7 P aeb2d3dfb23345e08d3af06cff7d5176: Deleting consensus metadata
I20260809 16:30:28.044806 15456 catalog_manager.cc:4841] TS a419bc3e42cb42b2a9b0201e1b5f72a0 (127.30.23.131:35737): tablet bfdc74c3e2b34695b47bb5950ad4beb7 (table test-table [id=695113444b2d47f99ed0b48bc389888a]) successfully deleted
I20260809 16:30:28.045286 15461 catalog_manager.cc:4841] TS aeb2d3dfb23345e08d3af06cff7d5176 (127.30.23.130:43905): tablet bfdc74c3e2b34695b47bb5950ad4beb7 (table test-table [id=695113444b2d47f99ed0b48bc389888a]) successfully deleted
W20260809 16:30:28.045569 15456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:43993 (error 108)
I20260809 16:30:28.045933 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 15817
I20260809 16:30:28.050691 16249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:30:28.050524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:43993 (local address 127.0.0.1:57792)
0809 16:30:28.050596 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 16:30:28.050655 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:43993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 16:30:28.050855 15456 catalog_manager.cc:4584] TS 8f298628a19942269b24ed5dcb9f2f7a (127.30.23.129:43993): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bfdc74c3e2b34695b47bb5950ad4beb7: Network error: Client connection negotiation failed: client connection to 127.30.23.129:43993: connect: Connection refused (error 111)
I20260809 16:30:28.054296 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 16032
I20260809 16:30:28.071780 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 15427
2026-08-09T16:30:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2325 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:30:28 dist-test-slave-j2vl krb5kdc[16300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:30:28 dist-test-slave-j2vl krb5kdc[16300](info): set up 2 sockets
Aug 09 16:30:28 dist-test-slave-j2vl krb5kdc[16300](info): commencing operation
krb5kdc: starting...
W20260809 16:30:29.409173 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:30:29 dist-test-slave-j2vl krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:30:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:30:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.23.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.23.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/rangeradmin_127.30.23.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/rangeradmin_127.30.23.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.23.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.23.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/rangerlookup_127.30.23.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/rangerlookup_127.30.23.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.23.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.23.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/HTTP_127.30.23.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/HTTP_127.30.23.148@KRBTEST.COM.keytab.
I20260809 16:30:29.598555 30814 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskunTR4f/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/postgres -l logfile start
2026-08-09 16:30:32.634 UTC [16346] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 16:30:32.634 UTC [16346] LOG: listening on IPv4 address "127.30.23.148", port 49455
2026-08-09 16:30:32.638 UTC [16346] LOG: listening on Unix socket "/tmp/.s.PGSQL.49455"
2026-08-09 16:30:32.664 UTC [16351] LOG: database system was shut down at 2026-08-09 16:30:30 UTC
2026-08-09 16:30:32.671 UTC [16346] LOG: database system is ready to accept connections
I20260809 16:30:33.892721 30814 mini_postgres.cc:96] Postgres bound to 49455
2026-08-09 16:30:33.900 UTC [16365] FATAL: database "slave" does not exist
127.30.23.148:49455 - accepting connections
I20260809 16:30:33.901851 30814 mini_ranger.cc:161] Starting Ranger...
I20260809 16:30:33.919791 30814 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 16:30:34.639766 30814 mini_ranger.cc:87] Created ranger Postgres database
I20260809 16:30:34.639871 30814 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-admin
2026-08-09 16:30:34,996 [I] DB FLAVOR :POSTGRES
2026-08-09 16:30:34,997 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:30:34,997 [I] Checking connection..
2026-08-09 16:30:34,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 16:30:35,282 [I] Checking connection passed.
2026-08-09 16:30:35,283 [I] --------- Verifying version history table ---------
2026-08-09 16:30:35,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:30:35,564 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:30:35,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:30:35,849 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:30:35,849 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 16:30:35,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 16:30:36.131 UTC [16473] WARNING: there is no transaction in progress
2026-08-09 16:30:36,141 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 16:30:36,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:30:36,416 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 16:30:36,416 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:30:36,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 16:30:36,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 16:30:36,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-j2vl.c.gcp-upstream.internal','N') ;"
2026-08-09 16:30:37,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:30:37,467 [I] Table x_portal_user does not exist in database ranger
2026-08-09 16:30:37,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:30:37,743 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 16:30:37,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:30:38,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:30:38,297 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 16:30:38,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 16:30:39.498 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.518 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.540 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.560 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.580 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.601 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.664 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.673 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.691 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.703 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.717 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.734 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.745 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.753 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.762 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:39.772 UTC [16648] WARNING: there is no transaction in progress
2026-08-09 16:30:40,625 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 16:30:40,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 16:30:40,881 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 16:30:40,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:30:41,145 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 16:30:41,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:30:41,442 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 16:30:41,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:30:41,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:30:41,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-j2vl.c.gcp-upstream.internal';"
2026-08-09 16:30:42,272 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 16:30:42,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 16:30:42,530 [I] DB_PATCHES have already been applied
I20260809 16:30:42.536147 30814 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-admin:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskunTR4f/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 16:30:42.536226 30814 mini_ranger.cc:193] Using host: 127.30.23.148
I20260809 16:30:42.540385 30814 mini_ranger.cc:227] Ranger admin URL: http://127.30.23.148:38165
Aug 09, 2026 4:30:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:30:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:30:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:30:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:30:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:30:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:30:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 16:30:43.946679 30814 mini_ranger.cc:160] Time spent starting Ranger: real 10.045s user 0.000s sys 0.012s
Aug 09, 2026 4:31:06 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:31:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 09, 2026 4:31:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:31:06 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:31:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:31:06 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:31:07 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 16:31:07.775455 30814 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.23.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.23.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/rangerkms_127.30.23.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/rangerkms_127.30.23.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.23.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.23.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.23.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/HTTP_127.30.23.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.23.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/HTTP_127.30.23.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 16:31:07 dist-test-slave-j2vl krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293067, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 16:31:07.881778 30814 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 16:31:07.894172 30814 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 16:31:08.991266 30814 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 16:31:08.991350 30814 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-kms
I20260809 16:31:08.991364 30814 mini_ranger_kms.cc:227] Using postgres at 127.30.23.148:49455
2026-08-09 16:31:09,335 [I] DB FLAVOR :POSTGRES
2026-08-09 16:31:09,335 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:31:09,335 [I] Checking connection
2026-08-09 16:31:09,336 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 16:31:09,629 [I] connection success
2026-08-09 16:31:09,630 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 16:31:09,630 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 16:31:09,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 16:31:09,899 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 16:31:09,899 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:31:09,899 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 16:31:09,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:49455/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 16:31:10,184 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 16:31:10.747665 30814 mini_ranger_kms.cc:310] Created kms service
I20260809 16:31:10.945894 30814 mini_ranger_kms.cc:326] Created kudu user
I20260809 16:31:10.996887 30814 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 16:31:10 dist-test-slave-j2vl krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293067, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:31:11.449154 30814 mini_ranger_kms.cc:384] Added ranger policy
I20260809 16:31:11.449297 30814 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-kms:/tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 16:31:11.449321 30814 mini_ranger_kms.cc:240] Using host: 127.30.23.148
I20260809 16:31:11.454193 30814 mini_ranger_kms.cc:276] Ranger KMS PID: 16939
I20260809 16:31:11.454334 30814 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.23.148:32891
Aug 09, 2026 4:31:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:31:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:31:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:31:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:31:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:31:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 16:31:14 dist-test-slave-j2vl krb5kdc[16300](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786293074, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.23.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:31:14.294404 30814 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.413s user 0.002s sys 0.323s
I20260809 16:31:14.294533 30814 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 16:31:14.294626 30814 mini_ranger_kms.cc:401] 127.30.23.148:32891/kms/v1/keys
Aug 09 16:31:14 dist-test-slave-j2vl krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293074, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.23.148@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
Aug 09 16:31:15 dist-test-slave-j2vl krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293074, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.23.148@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
Aug 09 16:31:17 dist-test-slave-j2vl krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:31:17.544919 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:44777
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:44777
--ranger_config_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.23.148:32891/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:31:17.639335 17015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:31:17.639590 17015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:31:17.639614 17015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:31:17.640846 17015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:31:17.640910 17015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:31:17.640931 17015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:31:17.640945 17015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:31:17.640959 17015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:31:17.642740 17015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.23.148:32891/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:44777
--ranger_config_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:44777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:31:17.642971 17015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:31:17.643158 17015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:31:17.650667 17022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:31:17.650648 17026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:31:17.650652 17023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:31:17.650799 17015 server_base.cc:1034] running on GCE node
I20260809 16:31:17.651216 17015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:31:17.651572 17015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:31:17.652833 17015 hybrid_clock.cc:648] HybridClock initialized: now 1786293077652772 us; error 68 us; skew 500 ppm
Aug 09 16:31:17 dist-test-slave-j2vl krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293077, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:31:17.660620 17015 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:31:17.661554 17015 webserver.cc:460] Webserver started at http://127.30.23.190:45771/ using document root <none> and password file <none>
I20260809 16:31:17.661820 17015 fs_manager.cc:359] Metadata directory not provided
I20260809 16:31:17.661893 17015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:31:17.662061 17015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:31:17 dist-test-slave-j2vl krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293077, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:31:17.723129 17015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "1ae336832a734f06be5b4651f9bfec81"
format_stamp: "Formatted at 2026-08-09 16:31:17 on dist-test-slave-j2vl"
server_key: "f94c30aff59389be558341142420e15e"
server_key_iv: "7835ed5ca47f44801b333395d4469640"
server_key_version: "kuduclusterkey@0"
I20260809 16:31:17.723654 17015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "1ae336832a734f06be5b4651f9bfec81"
format_stamp: "Formatted at 2026-08-09 16:31:17 on dist-test-slave-j2vl"
server_key: "f94c30aff59389be558341142420e15e"
server_key_iv: "7835ed5ca47f44801b333395d4469640"
server_key_version: "kuduclusterkey@0"
I20260809 16:31:17.725750 17015 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260809 16:31:17.739941 17034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:31:17.740218 17015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:31:17.740350 17015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "1ae336832a734f06be5b4651f9bfec81"
format_stamp: "Formatted at 2026-08-09 16:31:17 on dist-test-slave-j2vl"
server_key: "f94c30aff59389be558341142420e15e"
server_key_iv: "7835ed5ca47f44801b333395d4469640"
server_key_version: "kuduclusterkey@0"
I20260809 16:31:17.740468 17015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:31:17.756457 17015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:31:17.759070 17015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:31:17.759263 17015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:31:17.763866 17015 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:44777
I20260809 16:31:17.763919 17123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:44777 every 8 connection(s)
I20260809 16:31:17.764319 17015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:31:17.764849 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 17015
I20260809 16:31:17.764974 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:31:17.765174 30814 external_mini_cluster.cc:1428] Setting key d3661a85dfb9a3947fa96b3e0e0acb74
I20260809 16:31:17.765168 17126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:31:17.769513 17126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81: Bootstrap starting.
I20260809 16:31:17.770491 17126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81: Neither blocks nor log segments found. Creating new log.
I20260809 16:31:17.770903 17126 log.cc:824] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81: Log is configured to *not* fsync() on all Append() calls
I20260809 16:31:17.771932 17126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81: No bootstrap required, opened a new log
Aug 09 16:31:17 dist-test-slave-j2vl krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:31:17.775730 17126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae336832a734f06be5b4651f9bfec81" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44777 } }
I20260809 16:31:17.775955 17126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:31:17.775993 17126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ae336832a734f06be5b4651f9bfec81, State: Initialized, Role: FOLLOWER
I20260809 16:31:17.776134 17126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae336832a734f06be5b4651f9bfec81" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44777 } }
I20260809 16:31:17.776203 17126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:31:17.776232 17126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:31:17.776273 17126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:31:17.777892 17126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae336832a734f06be5b4651f9bfec81" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44777 } }
I20260809 16:31:17.778079 17126 leader_election.cc:302] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ae336832a734f06be5b4651f9bfec81; no voters:
I20260809 16:31:17.778163 17129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:17.766001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60590 (local address 127.30.23.190:44777)
0809 16:31:17.766213 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 16:31:17.766223 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:31:17.766260 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:31:17.766694 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:31:17.766701 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:31:17.766758 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:31:17.766968 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:31:17.767677 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:17.768690 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:17.769637 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:17.770112 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:17.770246 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:17.773754 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:31:17.773759 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:31:17.773759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:31:17.773773 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:31:17.775590 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:31:17.776787 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:31:17.776792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:31:17.776793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:31:17.776890 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:31:17.777135 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:31:17.777136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:31:17.777136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:31:17.777479 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:31:17.777634 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 16:31:17.777760 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 16:31:17.777893 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":108,"threads_started":1}
I20260809 16:31:17.778414 17126 leader_election.cc:288] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:31:17.778633 17134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:31:17.778719 17126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:31:17.778995 17134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [term 1 LEADER]: Becoming Leader. State: Replica: 1ae336832a734f06be5b4651f9bfec81, State: Running, Role: LEADER
I20260809 16:31:17.779340 17134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae336832a734f06be5b4651f9bfec81" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44777 } }
I20260809 16:31:17.779819 17134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ae336832a734f06be5b4651f9bfec81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae336832a734f06be5b4651f9bfec81" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44777 } } }
I20260809 16:31:17.780062 17134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [sys.catalog]: This master's current role is: LEADER
I20260809 16:31:17.780210 17134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ae336832a734f06be5b4651f9bfec81. Latest consensus state: current_term: 1 leader_uuid: "1ae336832a734f06be5b4651f9bfec81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae336832a734f06be5b4651f9bfec81" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44777 } } }
I20260809 16:31:17.780308 17148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:31:17.780333 17134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81 [sys.catalog]: This master's current role is: LEADER
I20260809 16:31:17.782615 17148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:31:17.782992 17126 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 16:31:17.787931 17148 catalog_manager.cc:1347] Generated new cluster ID: 4513ed629e904b0198055037e6286508
I20260809 16:31:17.788010 17148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:31:17.818874 17148 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:31:17.819804 17148 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:31:17.853118 17148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81: Generated new TSK 0
I20260809 16:31:17.853387 17148 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:31:18.605141 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.23.148:32891/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44777
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:31:18.734432 17201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:31:18.734670 17201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:31:18.734699 17201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:31:18.736194 17201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:31:18.736253 17201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:31:18.736284 17201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:31:18.738118 17201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.23.148:32891/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:31:18.738448 17201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:31:18.738725 17201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:31:18.744482 17217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:31:18.744477 17211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:31:18.745026 17209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:31:18.745617 17201 server_base.cc:1034] running on GCE node
I20260809 16:31:18.745841 17201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:31:18.746124 17201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:31:18.747326 17201 hybrid_clock.cc:648] HybridClock initialized: now 1786293078747292 us; error 54 us; skew 500 ppm
Aug 09 16:31:18 dist-test-slave-j2vl krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293078, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:31:18.750800 17201 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:31:18.751430 17201 webserver.cc:460] Webserver started at http://127.30.23.129:41753/ using document root <none> and password file <none>
I20260809 16:31:18.751735 17201 fs_manager.cc:359] Metadata directory not provided
I20260809 16:31:18.751806 17201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:31:18.752015 17201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:31:18 dist-test-slave-j2vl krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293078, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:31:18.772789 17201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "517df19021ba419792b14085f86cd7db"
format_stamp: "Formatted at 2026-08-09 16:31:18 on dist-test-slave-j2vl"
server_key: "ec06cbc014754a1c6833649b1a661452"
server_key_iv: "f8b000f0c1376b76ce36b777ef5cd898"
server_key_version: "kuduclusterkey@0"
I20260809 16:31:18.774998 17201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "517df19021ba419792b14085f86cd7db"
format_stamp: "Formatted at 2026-08-09 16:31:18 on dist-test-slave-j2vl"
server_key: "ec06cbc014754a1c6833649b1a661452"
server_key_iv: "f8b000f0c1376b76ce36b777ef5cd898"
server_key_version: "kuduclusterkey@0"
I20260809 16:31:18.777515 17201 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 16:31:18.795202 17226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:31:18.795709 17201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:31:18.795838 17201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "517df19021ba419792b14085f86cd7db"
format_stamp: "Formatted at 2026-08-09 16:31:18 on dist-test-slave-j2vl"
server_key: "ec06cbc014754a1c6833649b1a661452"
server_key_iv: "f8b000f0c1376b76ce36b777ef5cd898"
server_key_version: "kuduclusterkey@0"
I20260809 16:31:18.795977 17201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:31:18.808576 17201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:31:18.811754 17201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:31:18.812012 17201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:31:18.812405 17201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:31:18.812825 17201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:31:18.812881 17201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:31:18.812916 17201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:31:18.812968 17201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:31:18.822052 17201 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:33569
I20260809 16:31:18.822651 17201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:31:18.823601 17401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:33569 every 8 connection(s)
I20260809 16:31:18.825965 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 17201
I20260809 16:31:18.826090 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:31:18.826304 30814 external_mini_cluster.cc:1428] Setting key c62ce1ea3e5f603642194eb1304c3e78
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
Aug 09 16:31:18 dist-test-slave-j2vl krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293078, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
I20260809 16:31:18.852111 17406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:18.825143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:57417 (local address 127.30.23.190:44777)
0809 16:31:18.830665 (+ 5522us) server_negotiation.cc:206] Beginning negotiation
0809 16:31:18.830667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:31:18.830683 (+ 16us) server_negotiation.cc:407] Connection header received
0809 16:31:18.830716 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:31:18.830717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:31:18.830733 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:31:18.830844 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:31:18.831943 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:18.832765 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:18.842358 (+ 9593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:18.842708 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:18.842802 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:18.847421 (+ 4619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:31:18.847486 (+ 65us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:31:18.847487 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:31:18.847502 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:31:18.849815 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:31:18.850484 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:31:18.850486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:31:18.850486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:31:18.850565 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:31:18.850918 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:31:18.850920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:31:18.850920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:31:18.851076 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:31:18.851137 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:31:18.851896 (+ 759us) server_negotiation.cc:299] Negotiation successful
0809 16:31:18.851994 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5480,"thread_start_us":94,"threads_started":1}
I20260809 16:31:18.852859 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:18.825136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44777 (local address 127.30.23.129:57417)
0809 16:31:18.825366 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 16:31:18.825415 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:31:18.826211 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:31:18.831023 (+ 4812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:31:18.831030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:31:18.831397 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:31:18.831770 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:18.831780 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:18.832910 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:18.832911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:18.842170 (+ 9259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:18.842178 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:18.842931 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:18.842932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:18.843137 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:18.843856 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:31:18.843869 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:31:18.847238 (+ 3369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:31:18.849948 (+ 2710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:31:18.849961 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:31:18.849965 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:31:18.850342 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:31:18.850655 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:31:18.850656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:31:18.850657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:31:18.850835 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:31:18.851171 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:31:18.851185 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:31:18.851379 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 16:31:18.851493 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 16:31:18.852524 (+ 1031us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260809 16:31:18.855592 17404 heartbeater.cc:347] Connected to a master server at 127.30.23.190:44777
I20260809 16:31:18.855731 17404 heartbeater.cc:464] Registering TS with master...
I20260809 16:31:18.855966 17404 heartbeater.cc:511] Master 127.30.23.190:44777 requested a full tablet report, sending...
I20260809 16:31:18.856693 17060 ts_manager.cc:194] Registered new tserver with Master: 517df19021ba419792b14085f86cd7db (127.30.23.129:33569)
I20260809 16:31:18.857815 17060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:57417
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:31:18.870357 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.23.148:32891/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44777
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:31:19.006733 17412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:31:19.007206 17412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:31:19.007296 17412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:31:19.009826 17412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:31:19.010259 17412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:31:19.010363 17412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:31:19.013142 17412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.23.148:32891/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:31:19.013784 17412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:31:19.014150 17412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:31:19.021544 17424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:31:19.021993 17423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:31:19.021544 17428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:31:19.023171 17412 server_base.cc:1034] running on GCE node
I20260809 16:31:19.023779 17412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:31:19.024235 17412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:31:19.025417 17412 hybrid_clock.cc:648] HybridClock initialized: now 1786293079025394 us; error 42 us; skew 500 ppm
Aug 09 16:31:19 dist-test-slave-j2vl krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293079, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:31:19.029107 17412 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:31:19.029728 17412 webserver.cc:460] Webserver started at http://127.30.23.130:34389/ using document root <none> and password file <none>
I20260809 16:31:19.030019 17412 fs_manager.cc:359] Metadata directory not provided
I20260809 16:31:19.030076 17412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:31:19.030412 17412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:31:19 dist-test-slave-j2vl krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293079, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:31:19.046012 17412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "302b15db02f74301aed0c74e9beb1b42"
format_stamp: "Formatted at 2026-08-09 16:31:19 on dist-test-slave-j2vl"
server_key: "ba3c33de01316d61e4585dc764bb405a"
server_key_iv: "7f1026d719ebda7aa0f9fb5cafe316e0"
server_key_version: "kuduclusterkey@0"
I20260809 16:31:19.046649 17412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "302b15db02f74301aed0c74e9beb1b42"
format_stamp: "Formatted at 2026-08-09 16:31:19 on dist-test-slave-j2vl"
server_key: "ba3c33de01316d61e4585dc764bb405a"
server_key_iv: "7f1026d719ebda7aa0f9fb5cafe316e0"
server_key_version: "kuduclusterkey@0"
I20260809 16:31:19.052349 17412 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260809 16:31:19.075542 17439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
Aug 09 16:31:19 dist-test-slave-j2vl krb5kdc[16300](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786293079, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:31:19.075752 17412 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260809 16:31:19.075845 17412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "302b15db02f74301aed0c74e9beb1b42"
format_stamp: "Formatted at 2026-08-09 16:31:19 on dist-test-slave-j2vl"
server_key: "ba3c33de01316d61e4585dc764bb405a"
server_key_iv: "7f1026d719ebda7aa0f9fb5cafe316e0"
server_key_version: "kuduclusterkey@0"
I20260809 16:31:19.075963 17412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:31:19.097564 17412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:31:19.100513 17412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:31:19.100759 17412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:31:19.101028 17412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:31:19.101528 17412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:31:19.101630 17412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:31:19.101713 17412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:31:19.101774 17412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:31:19.112705 17412 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:46083
I20260809 16:31:19.113482 17412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:31:19.113997 17623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:46083 every 8 connection(s)
I20260809 16:31:19.121873 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 17412
I20260809 16:31:19.122031 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:31:19.122293 30814 external_mini_cluster.cc:1428] Setting key 901619f42b1b474bce7277ed4e916a70
Aug 09 16:31:19 dist-test-slave-j2vl krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293079, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:31:19.127357 17406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.115257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:60027 (local address 127.30.23.190:44777)
0809 16:31:19.115312 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:31:19.115315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:31:19.115721 (+ 406us) server_negotiation.cc:407] Connection header received
0809 16:31:19.116479 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:31:19.116481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:31:19.116499 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:31:19.116534 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:31:19.117599 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.118456 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.120357 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.120836 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.120899 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.123762 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:31:19.123764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:31:19.123764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:31:19.123773 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:31:19.125611 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:31:19.126176 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:31:19.126177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:31:19.126178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:31:19.126238 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:31:19.126552 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:31:19.126553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:31:19.126553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:31:19.126693 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:31:19.126735 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:31:19.127121 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 16:31:19.127222 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:31:19.128383 17626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.115236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44777 (local address 127.30.23.130:60027)
0809 16:31:19.115575 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 16:31:19.115623 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 16:31:19.116327 (+ 704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:31:19.116744 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:31:19.116749 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:31:19.117091 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:31:19.117471 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.117478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.118601 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.118602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.120225 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.120230 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.121016 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.121017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.121191 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.121908 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:31:19.121920 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:31:19.123647 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:31:19.125730 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:31:19.125740 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:31:19.125744 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:31:19.126082 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:31:19.126323 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:31:19.126323 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:31:19.126324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:31:19.126476 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:31:19.126781 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:31:19.126788 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:31:19.126969 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 16:31:19.127747 (+ 778us) client_negotiation.cc:240] Negotiation successful
0809 16:31:19.128094 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":108,"threads_started":1}
I20260809 16:31:19.129237 17624 heartbeater.cc:347] Connected to a master server at 127.30.23.190:44777
I20260809 16:31:19.129375 17624 heartbeater.cc:464] Registering TS with master...
I20260809 16:31:19.129603 17624 heartbeater.cc:511] Master 127.30.23.190:44777 requested a full tablet report, sending...
I20260809 16:31:19.130120 17060 ts_manager.cc:194] Registered new tserver with Master: 302b15db02f74301aed0c74e9beb1b42 (127.30.23.130:46083)
I20260809 16:31:19.130676 17060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:60027
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:31:19.177850 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.23.148:32891/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44777
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:31:19.319188 17630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:31:19.319496 17630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:31:19.319541 17630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:31:19.321606 17630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:31:19.321763 17630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:31:19.321885 17630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:31:19.324584 17630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.23.148:32891/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:44777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:31:19.325177 17630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:31:19.325608 17630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:31:19.334434 17630 server_base.cc:1034] running on GCE node
W20260809 16:31:19.337666 17639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:31:19.338469 17643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:31:19.341655 17640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:31:19.342108 17630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:31:19.342597 17630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:31:19.358206 17630 hybrid_clock.cc:648] HybridClock initialized: now 1786293079358134 us; error 96 us; skew 500 ppm
Aug 09 16:31:19 dist-test-slave-j2vl krb5kdc[16300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293079, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:31:19.362954 17630 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:31:19.364198 17630 webserver.cc:460] Webserver started at http://127.30.23.131:44409/ using document root <none> and password file <none>
I20260809 16:31:19.364832 17630 fs_manager.cc:359] Metadata directory not provided
I20260809 16:31:19.365025 17630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:31:19.365267 17630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:31:19 dist-test-slave-j2vl krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293079, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:31:19.393072 17630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "654024209d374ed2bc4dd2e7b4f1ba93"
format_stamp: "Formatted at 2026-08-09 16:31:19 on dist-test-slave-j2vl"
server_key: "ceb971222d47ec0dd8c0e2c0717f2574"
server_key_iv: "cccbbb9bfe453a62af4442fa73cdf86a"
server_key_version: "kuduclusterkey@0"
I20260809 16:31:19.393815 17630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "654024209d374ed2bc4dd2e7b4f1ba93"
format_stamp: "Formatted at 2026-08-09 16:31:19 on dist-test-slave-j2vl"
server_key: "ceb971222d47ec0dd8c0e2c0717f2574"
server_key_iv: "cccbbb9bfe453a62af4442fa73cdf86a"
server_key_version: "kuduclusterkey@0"
I20260809 16:31:19.398288 17630 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 16:31:19.426273 17650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:31:19.426602 17630 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260809 16:31:19.426720 17630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "654024209d374ed2bc4dd2e7b4f1ba93"
format_stamp: "Formatted at 2026-08-09 16:31:19 on dist-test-slave-j2vl"
server_key: "ceb971222d47ec0dd8c0e2c0717f2574"
server_key_iv: "cccbbb9bfe453a62af4442fa73cdf86a"
server_key_version: "kuduclusterkey@0"
I20260809 16:31:19.426837 17630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:31:19.461313 17630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:31:19.464330 17630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:31:19.464514 17630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:31:19.464762 17630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:31:19.465194 17630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:31:19.465250 17630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:31:19.465286 17630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:31:19.465323 17630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:31:19.477329 17630 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:44219
I20260809 16:31:19.477988 17630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:31:19.479498 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 17630
I20260809 16:31:19.479617 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:31:19.479823 30814 external_mini_cluster.cc:1428] Setting key e4935b08076dc627f2eac8ea5b550f5e
I20260809 16:31:19.479825 17831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:44219 every 8 connection(s)
Aug 09 16:31:19 dist-test-slave-j2vl krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293079, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:31:19.509691 17406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.479230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:44345 (local address 127.30.23.190:44777)
0809 16:31:19.479997 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0809 16:31:19.479999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:31:19.480013 (+ 14us) server_negotiation.cc:407] Connection header received
0809 16:31:19.488725 (+ 8712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:31:19.488728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:31:19.488746 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:31:19.488782 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:31:19.490513 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.491470 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.494097 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.494469 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.494525 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.498078 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:31:19.498080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:31:19.498080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:31:19.498091 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:31:19.499955 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:31:19.508395 (+ 8440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:31:19.508398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:31:19.508398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:31:19.508485 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:31:19.508805 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:31:19.508806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:31:19.508807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:31:19.508971 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:31:19.509027 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:31:19.509480 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 16:31:19.509593 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734}
I20260809 16:31:19.510375 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.478891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44777 (local address 127.30.23.131:44345)
0809 16:31:19.479124 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 16:31:19.479183 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 16:31:19.488497 (+ 9314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:31:19.489514 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:31:19.489520 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:31:19.489874 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:31:19.490338 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.490347 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.492371 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.492373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.493918 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.493923 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.494648 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.494649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.494865 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.495654 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:31:19.495667 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:31:19.497913 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:31:19.507719 (+ 9806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:31:19.507737 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:31:19.507742 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:31:19.508226 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:31:19.508563 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:31:19.508564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:31:19.508564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:31:19.508722 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:31:19.509061 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:31:19.509069 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:31:19.509257 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 16:31:19.509864 (+ 607us) client_negotiation.cc:240] Negotiation successful
0809 16:31:19.510105 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":123,"threads_started":1}
I20260809 16:31:19.510871 17832 heartbeater.cc:347] Connected to a master server at 127.30.23.190:44777
I20260809 16:31:19.511005 17832 heartbeater.cc:464] Registering TS with master...
I20260809 16:31:19.511231 17832 heartbeater.cc:511] Master 127.30.23.190:44777 requested a full tablet report, sending...
I20260809 16:31:19.511631 17060 ts_manager.cc:194] Registered new tserver with Master: 654024209d374ed2bc4dd2e7b4f1ba93 (127.30.23.131:44219)
I20260809 16:31:19.512257 17060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:44345
I20260809 16:31:19.513949 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:31:19.536731 17406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.518139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60600 (local address 127.30.23.190:44777)
0809 16:31:19.518196 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:31:19.518198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:31:19.518561 (+ 363us) server_negotiation.cc:407] Connection header received
0809 16:31:19.523123 (+ 4562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:31:19.523126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:31:19.523147 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:31:19.523198 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:31:19.526925 (+ 3727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.527784 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.529499 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.529834 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.529889 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.531999 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:31:19.532001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:31:19.532002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:31:19.532012 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:31:19.534172 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:31:19.535646 (+ 1474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:31:19.535648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:31:19.535648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:31:19.535729 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:31:19.535988 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:31:19.535989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:31:19.535990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:31:19.536155 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:31:19.536271 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 16:31:19.536510 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 16:31:19.536613 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:31:19.543435 17060 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:31:19.546003 17060 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:31:19.579849 17857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.551903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:33569 (local address 127.0.0.1:58328)
0809 16:31:19.556465 (+ 4562us) negotiation.cc:107] Waiting for socket to connect
0809 16:31:19.556528 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 16:31:19.556609 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:31:19.557835 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:31:19.557836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:31:19.557838 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:31:19.557874 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.557875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.570862 (+ 12987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.570863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.572123 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.572127 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.579387 (+ 7260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.579388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.579543 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.579562 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 16:31:19.579665 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 16:31:19.579735 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2017,"thread_start_us":88,"threads_started":1}
I20260809 16:31:19.583746 17861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.552142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:44219 (local address 127.0.0.1:51082)
0809 16:31:19.557160 (+ 5018us) negotiation.cc:107] Waiting for socket to connect
0809 16:31:19.557171 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:31:19.557219 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:31:19.557645 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:31:19.557649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:31:19.557652 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:31:19.557750 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.557752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.568874 (+ 11122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.568876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.570437 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.570441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.583327 (+ 12886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.583328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.583410 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.583415 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:31:19.583495 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 16:31:19.583639 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4999,"thread_start_us":75,"threads_started":1}
I20260809 16:31:19.585529 17856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.553016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:46083 (local address 127.0.0.1:41426)
0809 16:31:19.564279 (+ 11263us) negotiation.cc:107] Waiting for socket to connect
0809 16:31:19.564296 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 16:31:19.564390 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:31:19.565550 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:31:19.565551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:31:19.565554 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:31:19.565617 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.565619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.566931 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.566933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.568351 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.568356 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.585186 (+ 16830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.585188 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.585262 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.585269 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:31:19.585377 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 16:31:19.585440 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11233,"thread_start_us":70,"threads_started":1}
I20260809 16:31:19.586127 17859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.552695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.30.23.131:44219)
0809 16:31:19.557097 (+ 4402us) server_negotiation.cc:206] Beginning negotiation
0809 16:31:19.557101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:31:19.557402 (+ 301us) server_negotiation.cc:407] Connection header received
0809 16:31:19.557447 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:31:19.557449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:31:19.557465 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:31:19.557536 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:31:19.557951 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.568750 (+ 10799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.575500 (+ 6750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.583042 (+ 7542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.583150 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.583266 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 16:31:19.585850 (+ 2584us) server_negotiation.cc:299] Negotiation successful
0809 16:31:19.585960 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4361,"thread_start_us":105,"threads_started":1}
I20260809 16:31:19.586663 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.552479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58328 (local address 127.30.23.129:33569)
0809 16:31:19.556783 (+ 4304us) server_negotiation.cc:206] Beginning negotiation
0809 16:31:19.556789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:31:19.556801 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:31:19.556840 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:31:19.556843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:31:19.556860 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:31:19.556939 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:31:19.558845 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.570757 (+ 11912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.577237 (+ 6480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.578212 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.578332 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.578430 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 16:31:19.586514 (+ 8084us) server_negotiation.cc:299] Negotiation successful
0809 16:31:19.586583 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4262,"thread_start_us":66,"threads_started":1}
I20260809 16:31:19.587061 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.553280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41426 (local address 127.30.23.130:46083)
0809 16:31:19.557377 (+ 4097us) server_negotiation.cc:206] Beginning negotiation
0809 16:31:19.557381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:31:19.564475 (+ 7094us) server_negotiation.cc:407] Connection header received
0809 16:31:19.564539 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:31:19.564542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:31:19.564562 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:31:19.564642 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:31:19.565714 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.566804 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.573490 (+ 6686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.584957 (+ 11467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.585043 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.585135 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 16:31:19.586906 (+ 1771us) server_negotiation.cc:299] Negotiation successful
0809 16:31:19.586969 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4031,"thread_start_us":101,"threads_started":1}
I20260809 16:31:19.590015 17275 tablet_service.cc:1467] Processing CreateTablet for tablet b20c63e39e2841dbbc5ee1a9e2293200 (DEFAULT_TABLE table=test-table [id=35b7b65da7d74777984e9eeb88f697b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:31:19.590487 17275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b20c63e39e2841dbbc5ee1a9e2293200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:31:19.596307 17518 tablet_service.cc:1467] Processing CreateTablet for tablet b20c63e39e2841dbbc5ee1a9e2293200 (DEFAULT_TABLE table=test-table [id=35b7b65da7d74777984e9eeb88f697b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:31:19.596757 17518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b20c63e39e2841dbbc5ee1a9e2293200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:31:19.602696 17727 tablet_service.cc:1467] Processing CreateTablet for tablet b20c63e39e2841dbbc5ee1a9e2293200 (DEFAULT_TABLE table=test-table [id=35b7b65da7d74777984e9eeb88f697b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:31:19.603150 17727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b20c63e39e2841dbbc5ee1a9e2293200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:31:19.613162 17865 tablet_bootstrap.cc:492] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42: Bootstrap starting.
I20260809 16:31:19.615080 17867 tablet_bootstrap.cc:492] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93: Bootstrap starting.
I20260809 16:31:19.615408 17864 tablet_bootstrap.cc:492] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db: Bootstrap starting.
I20260809 16:31:19.616724 17864 tablet_bootstrap.cc:654] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db: Neither blocks nor log segments found. Creating new log.
I20260809 16:31:19.617825 17867 tablet_bootstrap.cc:654] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93: Neither blocks nor log segments found. Creating new log.
I20260809 16:31:19.618405 17865 tablet_bootstrap.cc:654] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42: Neither blocks nor log segments found. Creating new log.
I20260809 16:31:19.618940 17865 log.cc:824] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42: Log is configured to *not* fsync() on all Append() calls
I20260809 16:31:19.619437 17864 log.cc:824] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db: Log is configured to *not* fsync() on all Append() calls
I20260809 16:31:19.619936 17867 log.cc:824] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93: Log is configured to *not* fsync() on all Append() calls
I20260809 16:31:19.621894 17865 tablet_bootstrap.cc:492] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42: No bootstrap required, opened a new log
I20260809 16:31:19.622076 17865 ts_tablet_manager.cc:1397] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260809 16:31:19.624680 17865 raft_consensus.cc:348] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302b15db02f74301aed0c74e9beb1b42" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46083 } } peers { permanent_uuid: "517df19021ba419792b14085f86cd7db" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33569 } } peers { permanent_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 44219 } }
I20260809 16:31:19.625083 17865 raft_consensus.cc:374] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:31:19.625190 17865 raft_consensus.cc:729] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 302b15db02f74301aed0c74e9beb1b42, State: Initialized, Role: FOLLOWER
I20260809 16:31:19.625526 17865 consensus_queue.cc:260] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302b15db02f74301aed0c74e9beb1b42" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46083 } } peers { permanent_uuid: "517df19021ba419792b14085f86cd7db" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33569 } } peers { permanent_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 44219 } }
I20260809 16:31:19.626621 17865 ts_tablet_manager.cc:1428] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 16:31:19.627346 17864 tablet_bootstrap.cc:492] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db: No bootstrap required, opened a new log
I20260809 16:31:19.627518 17864 ts_tablet_manager.cc:1397] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260809 16:31:19.632237 17867 tablet_bootstrap.cc:492] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93: No bootstrap required, opened a new log
I20260809 16:31:19.632481 17867 ts_tablet_manager.cc:1397] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20260809 16:31:19.635341 17867 raft_consensus.cc:348] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302b15db02f74301aed0c74e9beb1b42" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46083 } } peers { permanent_uuid: "517df19021ba419792b14085f86cd7db" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33569 } } peers { permanent_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 44219 } }
I20260809 16:31:19.635720 17867 raft_consensus.cc:374] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:31:19.635833 17867 raft_consensus.cc:729] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 654024209d374ed2bc4dd2e7b4f1ba93, State: Initialized, Role: FOLLOWER
I20260809 16:31:19.636152 17864 raft_consensus.cc:348] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302b15db02f74301aed0c74e9beb1b42" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46083 } } peers { permanent_uuid: "517df19021ba419792b14085f86cd7db" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33569 } } peers { permanent_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 44219 } }
I20260809 16:31:19.636505 17867 consensus_queue.cc:260] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302b15db02f74301aed0c74e9beb1b42" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46083 } } peers { permanent_uuid: "517df19021ba419792b14085f86cd7db" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33569 } } peers { permanent_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 44219 } }
I20260809 16:31:19.636814 17624 heartbeater.cc:503] Master 127.30.23.190:44777 was elected leader, sending a full tablet report...
I20260809 16:31:19.637063 17867 ts_tablet_manager.cc:1428] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 16:31:19.637177 17864 raft_consensus.cc:374] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:31:19.637259 17864 raft_consensus.cc:729] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 517df19021ba419792b14085f86cd7db, State: Initialized, Role: FOLLOWER
I20260809 16:31:19.637457 17864 consensus_queue.cc:260] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302b15db02f74301aed0c74e9beb1b42" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46083 } } peers { permanent_uuid: "517df19021ba419792b14085f86cd7db" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33569 } } peers { permanent_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 44219 } }
I20260809 16:31:19.637939 17832 heartbeater.cc:503] Master 127.30.23.190:44777 was elected leader, sending a full tablet report...
I20260809 16:31:19.637773 17864 ts_tablet_manager.cc:1428] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260809 16:31:19.638867 17404 heartbeater.cc:503] Master 127.30.23.190:44777 was elected leader, sending a full tablet report...
I20260809 16:31:19.648123 17872 raft_consensus.cc:482] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:31:19.648378 17872 raft_consensus.cc:504] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302b15db02f74301aed0c74e9beb1b42" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46083 } } peers { permanent_uuid: "517df19021ba419792b14085f86cd7db" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33569 } } peers { permanent_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 44219 } }
I20260809 16:31:19.653632 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.649084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:46083 (local address 127.30.23.131:48385)
0809 16:31:19.649149 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 16:31:19.649166 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 16:31:19.649234 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:31:19.649483 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:31:19.649485 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:31:19.649486 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:31:19.649563 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.649565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.650709 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.650710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.652290 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.652297 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.653242 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.653244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.653414 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.653429 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:31:19.653480 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:31:19.653534 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 16:31:19.654259 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.649222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:48385 (local address 127.30.23.130:46083)
0809 16:31:19.649266 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:31:19.649268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:31:19.649287 (+ 19us) server_negotiation.cc:407] Connection header received
0809 16:31:19.649317 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:31:19.649318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:31:19.649329 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:31:19.649413 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:31:19.649647 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.650567 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.652464 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.653097 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.653187 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.653198 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:31:19.654127 (+ 929us) server_negotiation.cc:299] Negotiation successful
0809 16:31:19.654186 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:31:19.654580 17548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b20c63e39e2841dbbc5ee1a9e2293200" candidate_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302b15db02f74301aed0c74e9beb1b42" is_pre_election: true
I20260809 16:31:19.654781 17548 raft_consensus.cc:2454] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654024209d374ed2bc4dd2e7b4f1ba93 in term 0.
I20260809 16:31:19.655326 17655 leader_election.cc:302] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 302b15db02f74301aed0c74e9beb1b42, 654024209d374ed2bc4dd2e7b4f1ba93; no voters:
I20260809 16:31:19.665403 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.655916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:33569 (local address 127.30.23.131:54963)
0809 16:31:19.656161 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 16:31:19.656174 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:31:19.656226 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:31:19.656819 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:31:19.656820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:31:19.656823 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:31:19.656874 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.656876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.657925 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.657926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.663905 (+ 5979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:31:19.663913 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.665096 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.665097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:31:19.665170 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.665173 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:31:19.665226 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 16:31:19.665290 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260809 16:31:19.665899 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.656069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:54963 (local address 127.30.23.129:33569)
0809 16:31:19.656122 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:31:19.656125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:31:19.656289 (+ 164us) server_negotiation.cc:407] Connection header received
0809 16:31:19.656323 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:31:19.656324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:31:19.656334 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:31:19.656386 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:31:19.656981 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.657809 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.664093 (+ 6284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.664926 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.665000 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.665012 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 16:31:19.665767 (+ 755us) server_negotiation.cc:299] Negotiation successful
0809 16:31:19.665824 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:31:19.666534 17324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b20c63e39e2841dbbc5ee1a9e2293200" candidate_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "517df19021ba419792b14085f86cd7db" is_pre_election: true
I20260809 16:31:19.666716 17324 raft_consensus.cc:2454] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654024209d374ed2bc4dd2e7b4f1ba93 in term 0.
I20260809 16:31:19.667063 17872 leader_election.cc:288] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 302b15db02f74301aed0c74e9beb1b42 (127.30.23.130:46083), 517df19021ba419792b14085f86cd7db (127.30.23.129:33569)
I20260809 16:31:19.667253 17872 raft_consensus.cc:2781] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:31:19.667397 17872 raft_consensus.cc:482] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:31:19.667487 17872 raft_consensus.cc:3037] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:31:19.668625 17872 raft_consensus.cc:504] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302b15db02f74301aed0c74e9beb1b42" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46083 } } peers { permanent_uuid: "517df19021ba419792b14085f86cd7db" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33569 } } peers { permanent_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 44219 } }
I20260809 16:31:19.669072 17548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b20c63e39e2841dbbc5ee1a9e2293200" candidate_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302b15db02f74301aed0c74e9beb1b42"
I20260809 16:31:19.669173 17548 raft_consensus.cc:3037] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:31:19.670537 17548 raft_consensus.cc:2454] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654024209d374ed2bc4dd2e7b4f1ba93 in term 1.
I20260809 16:31:19.671552 17655 leader_election.cc:302] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 302b15db02f74301aed0c74e9beb1b42, 654024209d374ed2bc4dd2e7b4f1ba93; no voters:
I20260809 16:31:19.672233 17324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b20c63e39e2841dbbc5ee1a9e2293200" candidate_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "517df19021ba419792b14085f86cd7db"
I20260809 16:31:19.672346 17324 raft_consensus.cc:3037] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:31:19.673749 17324 raft_consensus.cc:2454] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654024209d374ed2bc4dd2e7b4f1ba93 in term 1.
I20260809 16:31:19.674139 17872 leader_election.cc:288] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [CANDIDATE]: Term 1 election: Requested vote from peers 302b15db02f74301aed0c74e9beb1b42 (127.30.23.130:46083), 517df19021ba419792b14085f86cd7db (127.30.23.129:33569)
I20260809 16:31:19.674412 17872 raft_consensus.cc:2781] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:31:19.674546 17872 raft_consensus.cc:686] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [term 1 LEADER]: Becoming Leader. State: Replica: 654024209d374ed2bc4dd2e7b4f1ba93, State: Running, Role: LEADER
I20260809 16:31:19.674719 17872 consensus_queue.cc:237] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302b15db02f74301aed0c74e9beb1b42" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46083 } } peers { permanent_uuid: "517df19021ba419792b14085f86cd7db" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33569 } } peers { permanent_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 44219 } }
I20260809 16:31:19.675957 17060 catalog_manager.cc:5495] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 654024209d374ed2bc4dd2e7b4f1ba93 (127.30.23.131). New cstate: current_term: 1 leader_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302b15db02f74301aed0c74e9beb1b42" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "517df19021ba419792b14085f86cd7db" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "654024209d374ed2bc4dd2e7b4f1ba93" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 44219 } health_report { overall_health: HEALTHY } } }
Aug 09 16:31:19 dist-test-slave-j2vl krb5kdc[16300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293079, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:31:19.713750 17859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.709719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51086 (local address 127.30.23.131:44219)
0809 16:31:19.709771 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:31:19.709773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:31:19.709782 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:31:19.709835 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:31:19.709836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:31:19.709862 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:31:19.709958 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:31:19.710203 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.710995 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.712350 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:31:19.712667 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:31:19.712743 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:31:19.713007 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:31:19.713162 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:31:19.713247 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 16:31:19.713582 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 16:31:19.713609 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:31:19.725462 17324 raft_consensus.cc:1261] T b20c63e39e2841dbbc5ee1a9e2293200 P 517df19021ba419792b14085f86cd7db [term 1 FOLLOWER]: Refusing update from remote peer 654024209d374ed2bc4dd2e7b4f1ba93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:31:19.726066 17875 consensus_queue.cc:1035] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "517df19021ba419792b14085f86cd7db" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:31:19.729501 17548 raft_consensus.cc:1261] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42 [term 1 FOLLOWER]: Refusing update from remote peer 654024209d374ed2bc4dd2e7b4f1ba93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:31:19.730010 17875 consensus_queue.cc:1035] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "302b15db02f74301aed0c74e9beb1b42" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 16:31:19.732003 17834 tablet.cc:2369] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:31:19.747875 17060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:31:19.748013 17060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:31:19.749207 17060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ae336832a734f06be5b4651f9bfec81: Sending DeleteTablet for 3 replicas of tablet b20c63e39e2841dbbc5ee1a9e2293200
I20260809 16:31:19.749518 17881 mvcc.cc:205] Tried to move back new op lower bound from 7316656454550044672 to 7316656454349012992. Current Snapshot: MvccSnapshot[applied={T|T < 7316656454550044672}]
I20260809 16:31:19.750087 17727 tablet_service.cc:1514] Processing DeleteTablet for tablet b20c63e39e2841dbbc5ee1a9e2293200 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:31:19 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:51082
I20260809 16:31:19.750310 17275 tablet_service.cc:1514] Processing DeleteTablet for tablet b20c63e39e2841dbbc5ee1a9e2293200 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:31:19 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:58328
I20260809 16:31:19.750360 17518 tablet_service.cc:1514] Processing DeleteTablet for tablet b20c63e39e2841dbbc5ee1a9e2293200 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:31:19 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:41426
I20260809 16:31:19.750959 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 17201
I20260809 16:31:19.765805 17879 mvcc.cc:205] Tried to move back new op lower bound from 7316656454550044672 to 7316656454349012992. Current Snapshot: MvccSnapshot[applied={T|T < 7316656454550044672 or (T in {7316656454550044672})}]
I20260809 16:31:19.768640 17892 tablet_replica.cc:330] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93: stopping tablet replica
I20260809 16:31:19.768889 17892 raft_consensus.cc:2229] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:31:19.769100 17892 raft_consensus.cc:2258] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:31:19.771812 17894 tablet_replica.cc:330] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42: stopping tablet replica
I20260809 16:31:19.772102 17894 raft_consensus.cc:2229] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:31:19.772302 17894 raft_consensus.cc:2258] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:31:19.773499 17894 ts_tablet_manager.cc:1905] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:31:19.779325 17892 ts_tablet_manager.cc:1905] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:31:19.782929 17892 ts_tablet_manager.cc:1918] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:31:19.783160 17892 log.cc:1196] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/b20c63e39e2841dbbc5ee1a9e2293200
I20260809 16:31:19.783571 17892 ts_tablet_manager.cc:1939] T b20c63e39e2841dbbc5ee1a9e2293200 P 654024209d374ed2bc4dd2e7b4f1ba93: Deleting consensus metadata
I20260809 16:31:19.784152 17038 catalog_manager.cc:4841] TS 654024209d374ed2bc4dd2e7b4f1ba93 (127.30.23.131:44219): tablet b20c63e39e2841dbbc5ee1a9e2293200 (table test-table [id=35b7b65da7d74777984e9eeb88f697b7]) successfully deleted
I20260809 16:31:19.785494 17894 ts_tablet_manager.cc:1918] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:31:19.785671 17894 log.cc:1196] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/b20c63e39e2841dbbc5ee1a9e2293200
I20260809 16:31:19.786041 17894 ts_tablet_manager.cc:1939] T b20c63e39e2841dbbc5ee1a9e2293200 P 302b15db02f74301aed0c74e9beb1b42: Deleting consensus metadata
I20260809 16:31:19.786631 17037 catalog_manager.cc:4841] TS 302b15db02f74301aed0c74e9beb1b42 (127.30.23.130:46083): tablet b20c63e39e2841dbbc5ee1a9e2293200 (table test-table [id=35b7b65da7d74777984e9eeb88f697b7]) successfully deleted
W20260809 16:31:19.809598 17038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:33569 (error 108)
I20260809 16:31:19.810509 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 17412
I20260809 16:31:19.869858 17856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.869708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:33569 (local address 127.0.0.1:58338)
0809 16:31:19.869769 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 16:31:19.869810 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:33569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 16:31:19.870210 17038 catalog_manager.cc:4584] TS 517df19021ba419792b14085f86cd7db (127.30.23.129:33569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b20c63e39e2841dbbc5ee1a9e2293200: Network error: Client connection negotiation failed: client connection to 127.30.23.129:33569: connect: Connection refused (error 111)
I20260809 16:31:19.922860 17856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.922707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:33569 (local address 127.0.0.1:58348)
0809 16:31:19.922771 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 16:31:19.922808 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:33569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 16:31:19.923403 17856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.923310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:33569 (local address 127.0.0.1:58354)
0809 16:31:19.923348 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 16:31:19.923374 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:33569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 16:31:19.954890 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 17630
I20260809 16:31:19.983404 17856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:31:19.983164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:33569 (local address 127.0.0.1:58360)
0809 16:31:19.983321 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:31:19.983350 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:33569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260809 16:31:19.983714 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 17015
I20260809 16:31:20.001798 30814 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 16:31:20.347259 30814 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 16:31:20.347400 30814 mini_ranger.cc:66] Stopping Ranger...
I20260809 16:31:20.692960 30814 mini_ranger.cc:68] Stopped Ranger
2026-08-09 16:31:20.693 UTC [16346] LOG: received smart shutdown request
2026-08-09 16:31:20.698 UTC [16346] LOG: background worker "logical replication launcher" (PID 16362) exited with exit code 1
2026-08-09 16:31:20.699 UTC [16353] LOG: shutting down
2026-08-09 16:31:20.934 UTC [16346] LOG: database system is shut down
2026-08-09T16:31:20Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52961 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:31:21 dist-test-slave-j2vl krb5kdc[17909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:31:21 dist-test-slave-j2vl krb5kdc[17909](info): set up 2 sockets
Aug 09 16:31:21 dist-test-slave-j2vl krb5kdc[17909](info): commencing operation
krb5kdc: starting...
W20260809 16:31:22.349576 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:31:22 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:31:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:31:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.23.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.23.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/rangeradmin_127.30.23.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/rangeradmin_127.30.23.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.23.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.23.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/rangerlookup_127.30.23.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/rangerlookup_127.30.23.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.23.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.23.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/HTTP_127.30.23.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/HTTP_127.30.23.148@KRBTEST.COM.keytab.
I20260809 16:31:22.584378 30814 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskunTR4f/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/postgres -l logfile start
2026-08-09 16:31:25.495 UTC [17961] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 16:31:25.495 UTC [17961] LOG: listening on IPv4 address "127.30.23.148", port 46265
2026-08-09 16:31:25.499 UTC [17961] LOG: listening on Unix socket "/tmp/.s.PGSQL.46265"
2026-08-09 16:31:25.522 UTC [17965] LOG: database system was shut down at 2026-08-09 16:31:23 UTC
2026-08-09 16:31:25.526 UTC [17961] LOG: database system is ready to accept connections
I20260809 16:31:26.738368 30814 mini_postgres.cc:96] Postgres bound to 46265
2026-08-09 16:31:26.746 UTC [17976] FATAL: database "slave" does not exist
127.30.23.148:46265 - accepting connections
I20260809 16:31:26.747015 30814 mini_ranger.cc:161] Starting Ranger...
I20260809 16:31:26.763971 30814 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 16:31:27.356606 30814 mini_ranger.cc:87] Created ranger Postgres database
I20260809 16:31:27.356711 30814 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-admin
2026-08-09 16:31:27,637 [I] DB FLAVOR :POSTGRES
2026-08-09 16:31:27,638 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:31:27,638 [I] Checking connection..
2026-08-09 16:31:27,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 16:31:27,903 [I] Checking connection passed.
2026-08-09 16:31:27,904 [I] --------- Verifying version history table ---------
2026-08-09 16:31:27,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:31:28,184 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:31:28,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:31:28,441 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:31:28,442 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 16:31:28,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 16:31:28.721 UTC [18069] WARNING: there is no transaction in progress
2026-08-09 16:31:28,732 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 16:31:28,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:31:28,992 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 16:31:28,992 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:31:28,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 16:31:29,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 16:31:29,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-j2vl.c.gcp-upstream.internal','N') ;"
2026-08-09 16:31:29,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:31:30,086 [I] Table x_portal_user does not exist in database ranger
2026-08-09 16:31:30,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:31:30,336 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 16:31:30,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:31:30,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:31:30,867 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 16:31:30,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 16:31:31.976 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:31.997 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.017 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.036 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.055 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.075 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.139 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.148 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.167 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.179 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.191 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.207 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.218 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.226 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.234 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:32.243 UTC [18237] WARNING: there is no transaction in progress
2026-08-09 16:31:33,178 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 16:31:33,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 16:31:33,436 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 16:31:33,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:31:33,733 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 16:31:33,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:31:34,006 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 16:31:34,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:31:34,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:31:34,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-j2vl.c.gcp-upstream.internal';"
2026-08-09 16:31:34,824 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 16:31:34,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 16:31:35,089 [I] DB_PATCHES have already been applied
I20260809 16:31:35.095439 30814 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-admin:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskunTR4f/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 16:31:35.095522 30814 mini_ranger.cc:193] Using host: 127.30.23.148
I20260809 16:31:35.100185 30814 mini_ranger.cc:227] Ranger admin URL: http://127.30.23.148:45235
Aug 09, 2026 4:31:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:31:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:31:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:31:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:31:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:31:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:31:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 16:31:36.573148 30814 mini_ranger.cc:160] Time spent starting Ranger: real 9.826s user 0.000s sys 0.012s
Aug 09, 2026 4:31:58 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:31:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 09, 2026 4:31:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:31:59 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:31:59 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 16:32:00.658639 30814 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.23.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.23.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/rangerkms_127.30.23.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.23.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/rangerkms_127.30.23.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.23.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.23.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.23.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/HTTP_127.30.23.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.23.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/HTTP_127.30.23.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 16:32:00 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293120, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 16:32:00.758287 30814 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 16:32:00.771258 30814 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 16:32:02.004099 30814 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 16:32:02.004200 30814 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-kms
I20260809 16:32:02.004225 30814 mini_ranger_kms.cc:227] Using postgres at 127.30.23.148:46265
2026-08-09 16:32:02,197 [I] DB FLAVOR :POSTGRES
2026-08-09 16:32:02,197 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:32:02,197 [I] Checking connection
2026-08-09 16:32:02,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 16:32:02,454 [I] connection success
2026-08-09 16:32:02,454 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 16:32:02,454 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 16:32:02,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 16:32:02,729 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 16:32:02,730 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:32:02,730 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 16:32:02,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.23.148:46265/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 16:32:03,035 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 16:32:03.608299 30814 mini_ranger_kms.cc:310] Created kms service
I20260809 16:32:03.818856 30814 mini_ranger_kms.cc:326] Created kudu user
I20260809 16:32:03.875573 30814 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 16:32:03 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293120, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:32:04.357874 30814 mini_ranger_kms.cc:384] Added ranger policy
I20260809 16:32:04.358100 30814 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-kms:/tmp/dist-test-taskunTR4f/build/release/bin/postgresql.jar:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskunTR4f/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 16:32:04.358120 30814 mini_ranger_kms.cc:240] Using host: 127.30.23.148
I20260809 16:32:04.363350 30814 mini_ranger_kms.cc:276] Ranger KMS PID: 18522
I20260809 16:32:04.363468 30814 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.23.148:50931
Aug 09, 2026 4:32:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:32:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:32:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:32:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:32:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:32:06 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 16:32:07 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786293127, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.23.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:07.339103 30814 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.581s user 0.007s sys 0.170s
I20260809 16:32:07.339223 30814 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 16:32:07.339286 30814 mini_ranger_kms.cc:401] 127.30.23.148:50931/kms/v1/keys
Aug 09 16:32:07 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293127, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.23.148@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
Aug 09 16:32:07 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293127, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.23.148@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
Aug 09 16:32:10 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:32:10.396360 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:43727
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:34087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:43727
--ranger_config_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.23.148:50931/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:10.503501 18611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:10.503814 18611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:10.503857 18611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:10.505097 18611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:32:10.505198 18611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:10.505223 18611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:10.505244 18611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:32:10.505273 18611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:32:10.507010 18611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:34087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.23.148:50931/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:43727
--ranger_config_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:43727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:10.507318 18611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:10.507562 18611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:32:10.513265 18611 server_base.cc:1034] running on GCE node
W20260809 16:32:10.513280 18620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:10.513267 18619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:10.513602 18622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:10.513918 18611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:10.514271 18611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:10.515465 18611 hybrid_clock.cc:648] HybridClock initialized: now 1786293130515441 us; error 42 us; skew 500 ppm
Aug 09 16:32:10 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293130, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:10.521960 18611 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:32:10.522543 18611 webserver.cc:460] Webserver started at http://127.30.23.190:44563/ using document root <none> and password file <none>
I20260809 16:32:10.522815 18611 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:10.522888 18611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:10.523092 18611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:32:10 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293130, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:32:10.587345 18611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "6edcc300cfe542feb7353e2d9d6b067a"
format_stamp: "Formatted at 2026-08-09 16:32:10 on dist-test-slave-j2vl"
server_key: "ee6b466080acb8406883e7f48b68148a"
server_key_iv: "47752b0961bd201dfa2aa600b9293226"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:10.587776 18611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "6edcc300cfe542feb7353e2d9d6b067a"
format_stamp: "Formatted at 2026-08-09 16:32:10 on dist-test-slave-j2vl"
server_key: "ee6b466080acb8406883e7f48b68148a"
server_key_iv: "47752b0961bd201dfa2aa600b9293226"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:10.589440 18611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260809 16:32:10.605842 18633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:10.606084 18611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:32:10.606195 18611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "6edcc300cfe542feb7353e2d9d6b067a"
format_stamp: "Formatted at 2026-08-09 16:32:10 on dist-test-slave-j2vl"
server_key: "ee6b466080acb8406883e7f48b68148a"
server_key_iv: "47752b0961bd201dfa2aa600b9293226"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:10.606303 18611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:10.624440 18611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:10.626629 18611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:10.626818 18611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:10.632055 18611 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:43727
I20260809 16:32:10.632086 18711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:43727 every 8 connection(s)
I20260809 16:32:10.632501 18611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:32:10.633262 18713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:10.634915 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 18611
I20260809 16:32:10.635038 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:32:10.635253 30814 external_mini_cluster.cc:1428] Setting key c4416c4aaa86926a42a9cddea1423ea0
I20260809 16:32:10.636147 18713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a: Bootstrap starting.
I20260809 16:32:10.636927 18713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:10.637321 18713 log.cc:824] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:10.638985 18713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a: No bootstrap required, opened a new log
I20260809 16:32:10.641728 18713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6edcc300cfe542feb7353e2d9d6b067a" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43727 } }
I20260809 16:32:10.641969 18713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:10.642024 18713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6edcc300cfe542feb7353e2d9d6b067a, State: Initialized, Role: FOLLOWER
I20260809 16:32:10.642176 18713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6edcc300cfe542feb7353e2d9d6b067a" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43727 } }
I20260809 16:32:10.642285 18713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:32:10.642341 18713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:32:10.642416 18713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:10.643014 18713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6edcc300cfe542feb7353e2d9d6b067a" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43727 } }
I20260809 16:32:10.643139 18713 leader_election.cc:302] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6edcc300cfe542feb7353e2d9d6b067a; no voters:
I20260809 16:32:10.643335 18713 leader_election.cc:288] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:32:10.643366 18721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:32:10.643586 18721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 1 LEADER]: Becoming Leader. State: Replica: 6edcc300cfe542feb7353e2d9d6b067a, State: Running, Role: LEADER
I20260809 16:32:10.643708 18713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:32:10.643723 18721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6edcc300cfe542feb7353e2d9d6b067a" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43727 } }
Aug 09 16:32:10 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:10.644167 18721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6edcc300cfe542feb7353e2d9d6b067a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6edcc300cfe542feb7353e2d9d6b067a" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43727 } } }
I20260809 16:32:10.644357 18721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:10.644551 18721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6edcc300cfe542feb7353e2d9d6b067a. Latest consensus state: current_term: 1 leader_uuid: "6edcc300cfe542feb7353e2d9d6b067a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6edcc300cfe542feb7353e2d9d6b067a" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43727 } } }
I20260809 16:32:10.644656 18721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:10.644835 18735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:32:10.645560 18735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:32:10.645913 18713 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 16:32:10.653651 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:10.635984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55788 (local address 127.30.23.190:43727)
0809 16:32:10.636173 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:10.636184 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:10.636215 (+ 31us) server_negotiation.cc:407] Connection header received
0809 16:32:10.636791 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:10.636800 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:10.636863 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:10.637025 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:10.637860 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:10.639253 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:10.641044 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:10.641726 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:10.641844 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:10.644791 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:10.644795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:10.644796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:10.644809 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:10.651165 (+ 6356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:10.651721 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:10.651724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:10.651725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:10.651903 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:10.652129 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:10.652130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:10.652130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:10.652635 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:10.652842 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:10.652936 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 16:32:10.653308 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260809 16:32:10.654454 18735 catalog_manager.cc:1347] Generated new cluster ID: 77fdf41737fc42e3bafae858a3e2980c
I20260809 16:32:10.654515 18735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:32:10.680411 18735 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:32:10.681232 18735 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:32:10.695431 18735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a: Generated new TSK 0
I20260809 16:32:10.695742 18735 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:32:11.446861 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.23.148:50931/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43727
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:34087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:11.540503 18783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:11.540751 18783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:11.540793 18783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:11.542063 18783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:11.542135 18783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:11.542181 18783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:32:11.543972 18783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:34087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.23.148:50931/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:11.544325 18783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:11.544597 18783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:11.550343 18795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:11.550316 18792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:11.550851 18790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:11.551005 18783 server_base.cc:1034] running on GCE node
I20260809 16:32:11.551278 18783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:11.551578 18783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:11.552783 18783 hybrid_clock.cc:648] HybridClock initialized: now 1786293131552774 us; error 35 us; skew 500 ppm
Aug 09 16:32:11 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293131, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:11.555709 18783 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:32:11.556263 18783 webserver.cc:460] Webserver started at http://127.30.23.129:34211/ using document root <none> and password file <none>
I20260809 16:32:11.556545 18783 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:11.556618 18783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:11.556759 18783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:32:11 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293131, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:32:11.570524 18783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "504da40249894d4dacb0348b99731cf1"
format_stamp: "Formatted at 2026-08-09 16:32:11 on dist-test-slave-j2vl"
server_key: "de5d1536a71a46e9b47ae733f40666d2"
server_key_iv: "6242128ebf532053056d6350e5ec0b43"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:11.571015 18783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "504da40249894d4dacb0348b99731cf1"
format_stamp: "Formatted at 2026-08-09 16:32:11 on dist-test-slave-j2vl"
server_key: "de5d1536a71a46e9b47ae733f40666d2"
server_key_iv: "6242128ebf532053056d6350e5ec0b43"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:11.573742 18783 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 16:32:11.589197 18807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:11.589432 18783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 16:32:11.589548 18783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "504da40249894d4dacb0348b99731cf1"
format_stamp: "Formatted at 2026-08-09 16:32:11 on dist-test-slave-j2vl"
server_key: "de5d1536a71a46e9b47ae733f40666d2"
server_key_iv: "6242128ebf532053056d6350e5ec0b43"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:11.589668 18783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:11.619989 18783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:11.622655 18783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:11.622862 18783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:11.623180 18783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:11.623603 18783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:11.623664 18783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:11.623710 18783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:11.623739 18783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:11.631965 18783 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:42205
I20260809 16:32:11.632475 18783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:32:11.636126 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 18783
I20260809 16:32:11.636257 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:32:11.636444 30814 external_mini_cluster.cc:1428] Setting key f4773f1c8d306cc39e50cd19de2c4cf8
I20260809 16:32:11.634721 18998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:42205 every 8 connection(s)
Aug 09 16:32:11 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293131, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
I20260809 16:32:11.647492 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:11.635649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:39517 (local address 127.30.23.190:43727)
0809 16:32:11.636754 (+ 1105us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:11.636756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:11.636764 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:32:11.636788 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:11.636789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:11.636802 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:11.637158 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:11.638048 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:11.638826 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:11.640386 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:11.640608 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:11.640731 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:11.643571 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:11.643572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:11.643573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:11.643580 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:11.645659 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:11.646211 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:11.646212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:11.646213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:11.646297 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:11.646628 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:11.646629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:11.646629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:11.646784 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:11.646835 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:11.647277 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 16:32:11.647380 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1071,"thread_start_us":80,"threads_started":1}
I20260809 16:32:11.648087 19002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:11.635337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:43727 (local address 127.30.23.129:39517)
0809 16:32:11.635821 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:11.635876 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:11.636553 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:11.637321 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:11.637326 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:11.637617 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:11.637920 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:11.637927 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:11.638958 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:11.638959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:11.640239 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:11.640243 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:11.640758 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:11.640760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:11.640936 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:11.641643 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:11.641655 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:11.643442 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:11.645766 (+ 2324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:11.645775 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:11.645778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:11.646106 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:11.646381 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:11.646382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:11.646382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:11.646532 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:11.646890 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:11.646899 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:11.647085 (+ 186us) client_negotiation.cc:769] Sending connection context
0809 16:32:11.647650 (+ 565us) client_negotiation.cc:240] Negotiation successful
0809 16:32:11.647835 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":99,"threads_started":1}
I20260809 16:32:11.648728 18999 heartbeater.cc:347] Connected to a master server at 127.30.23.190:43727
I20260809 16:32:11.648835 18999 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:11.649050 18999 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
I20260809 16:32:11.649675 18657 ts_manager.cc:194] Registered new tserver with Master: 504da40249894d4dacb0348b99731cf1 (127.30.23.129:42205)
I20260809 16:32:11.650625 18657 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:39517
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:32:11.669473 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.23.148:50931/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43727
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:34087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:11.789005 19010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:11.789395 19010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:11.789466 19010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:11.791276 19010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:11.791508 19010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:11.791612 19010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:32:11.794131 19010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:34087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.23.148:50931/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:11.794410 19010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:11.794795 19010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:11.801709 19029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:11.801822 19010 server_base.cc:1034] running on GCE node
W20260809 16:32:11.802019 19025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:11.802069 19026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:11.802399 19010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:11.802740 19010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 09 16:32:11 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786293131, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:11.803941 19010 hybrid_clock.cc:648] HybridClock initialized: now 1786293131803913 us; error 40 us; skew 500 ppm
Aug 09 16:32:11 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293131, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:11.807513 19010 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:32:11.808127 19010 webserver.cc:460] Webserver started at http://127.30.23.130:43149/ using document root <none> and password file <none>
I20260809 16:32:11.808493 19010 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:11.808579 19010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:11.808787 19010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:32:11 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293131, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:32:11.824007 19010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "a1061ab3a87c40999929bcb1280e73dd"
format_stamp: "Formatted at 2026-08-09 16:32:11 on dist-test-slave-j2vl"
server_key: "b47d85dd8df96b1092ac338243bbcd23"
server_key_iv: "5fb1da30913485e1e676b82cb9ca9f3f"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:11.824479 19010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "a1061ab3a87c40999929bcb1280e73dd"
format_stamp: "Formatted at 2026-08-09 16:32:11 on dist-test-slave-j2vl"
server_key: "b47d85dd8df96b1092ac338243bbcd23"
server_key_iv: "5fb1da30913485e1e676b82cb9ca9f3f"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:11.833791 19010 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.004s
I20260809 16:32:11.851907 19043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:11.852299 19010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 16:32:11.852458 19010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "a1061ab3a87c40999929bcb1280e73dd"
format_stamp: "Formatted at 2026-08-09 16:32:11 on dist-test-slave-j2vl"
server_key: "b47d85dd8df96b1092ac338243bbcd23"
server_key_iv: "5fb1da30913485e1e676b82cb9ca9f3f"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:11.852571 19010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:11.870703 19010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:11.873203 19010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:11.873461 19010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:11.873783 19010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:11.874235 19010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:11.874279 19010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:11.874305 19010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:11.874343 19010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:11.883284 19010 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:34685
I20260809 16:32:11.883761 19010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:32:11.885510 19221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:34685 every 8 connection(s)
I20260809 16:32:11.887835 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 19010
I20260809 16:32:11.887964 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:32:11.888180 30814 external_mini_cluster.cc:1428] Setting key 9e57aff7a7d3413ab88619a86991e709
Aug 09 16:32:11 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293131, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:11.898453 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:11.886217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:58957 (local address 127.30.23.190:43727)
0809 16:32:11.886304 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:11.886306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:11.886546 (+ 240us) server_negotiation.cc:407] Connection header received
0809 16:32:11.887154 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:11.887156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:11.887172 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:11.887239 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:11.888014 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:11.888797 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:11.890500 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:11.890877 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:11.890930 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:11.894663 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:11.894666 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:11.894666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:11.894678 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:11.896457 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:11.897067 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:11.897068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:11.897069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:11.897132 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:11.897489 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:11.897490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:11.897490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:11.897657 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:11.897705 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:11.898178 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 16:32:11.898295 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:32:11.899118 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:11.886194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:43727 (local address 127.30.23.130:58957)
0809 16:32:11.886394 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:11.886443 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:11.887036 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:11.887308 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:11.887312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:11.887536 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:11.887881 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:11.887887 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:11.888930 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:11.888931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:11.890385 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:11.890388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:11.891057 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:11.891058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:11.891226 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:11.892576 (+ 1350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:11.892587 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:11.894525 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:11.896578 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:11.896586 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:11.896589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:11.896964 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:11.897212 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:11.897213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:11.897213 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:11.897396 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:11.897771 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:11.897777 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:11.897967 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 16:32:11.898573 (+ 606us) client_negotiation.cc:240] Negotiation successful
0809 16:32:11.898777 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":121,"threads_started":1}
I20260809 16:32:11.899859 19222 heartbeater.cc:347] Connected to a master server at 127.30.23.190:43727
I20260809 16:32:11.899978 19222 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:11.900209 19222 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
I20260809 16:32:11.900629 18657 ts_manager.cc:194] Registered new tserver with Master: a1061ab3a87c40999929bcb1280e73dd (127.30.23.130:34685)
I20260809 16:32:11.901155 18657 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:58957
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:32:11.928565 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.23.148:50931/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43727
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:34087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:12.027379 19232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:12.027765 19232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:12.027835 19232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:12.029598 19232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:12.029685 19232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:12.029728 19232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:32:12.032032 19232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:34087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.23.148:50931/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:12.032308 19232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:12.032662 19232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:12.039485 19248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:12.039620 19244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:12.039871 19250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:12.040640 19232 server_base.cc:1034] running on GCE node
I20260809 16:32:12.040899 19232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:12.041249 19232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:12.042445 19232 hybrid_clock.cc:648] HybridClock initialized: now 1786293132042420 us; error 31 us; skew 500 ppm
Aug 09 16:32:12 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:12.045917 19232 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:32:12.046494 19232 webserver.cc:460] Webserver started at http://127.30.23.131:33681/ using document root <none> and password file <none>
I20260809 16:32:12.046790 19232 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:12.046852 19232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:12.047003 19232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:32:12 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:32:12.072647 19232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "06dfe6fe35fe436db3eaaadedfc49984"
format_stamp: "Formatted at 2026-08-09 16:32:12 on dist-test-slave-j2vl"
server_key: "2a7a6394dbed4e7bea5a2b65f9d08614"
server_key_iv: "2ad9a8e7c8019a26e6ba0f723999233b"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:12.073119 19232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "06dfe6fe35fe436db3eaaadedfc49984"
format_stamp: "Formatted at 2026-08-09 16:32:12 on dist-test-slave-j2vl"
server_key: "2a7a6394dbed4e7bea5a2b65f9d08614"
server_key_iv: "2ad9a8e7c8019a26e6ba0f723999233b"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:12.075886 19232 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:32:12.091392 19261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:12.091594 19232 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:12.091698 19232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "06dfe6fe35fe436db3eaaadedfc49984"
format_stamp: "Formatted at 2026-08-09 16:32:12 on dist-test-slave-j2vl"
server_key: "2a7a6394dbed4e7bea5a2b65f9d08614"
server_key_iv: "2ad9a8e7c8019a26e6ba0f723999233b"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:12.091805 19232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:12.109723 19232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:12.112293 19232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:12.112437 19232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:12.112659 19232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:12.113042 19232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:12.113080 19232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:12.113111 19232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:12.113128 19232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:12.120414 19232 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:35917
I20260809 16:32:12.120775 19451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:35917 every 8 connection(s)
I20260809 16:32:12.120954 19232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:32:12.126194 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 19232
I20260809 16:32:12.126332 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:32:12.126543 30814 external_mini_cluster.cc:1428] Setting key 005049bef1c76451c070014fd3faac3e
Aug 09 16:32:12 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:12.133061 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:12.122104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:49249 (local address 127.30.23.190:43727)
0809 16:32:12.122152 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:12.122154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:12.122166 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:32:12.122936 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:12.122938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:12.122950 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:12.123033 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:12.123879 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:12.124670 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:12.126234 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:12.126560 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:12.126645 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:12.129555 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:12.129557 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:12.129557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:12.129567 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:12.131355 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:12.131886 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:12.131887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:12.131888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:12.131951 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:12.132255 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:12.132256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:12.132256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:12.132398 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:12.132456 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:12.132855 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 16:32:12.132958 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:32:12.133749 19454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:12.121784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:43727 (local address 127.30.23.131:49249)
0809 16:32:12.122024 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:12.122078 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:12.122797 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:12.123107 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:12.123111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:12.123428 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:12.123748 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:12.123755 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:12.124800 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:12.124801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:12.126103 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:12.126106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:12.126724 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:12.126725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:12.126886 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:12.127533 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:12.127545 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:12.129413 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:12.131469 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:12.131478 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:12.131481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:12.131791 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:12.132030 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:12.132030 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:12.132031 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:12.132173 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:12.132508 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:12.132515 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:12.132691 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 16:32:12.133217 (+ 526us) client_negotiation.cc:240] Negotiation successful
0809 16:32:12.133428 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":105,"threads_started":1}
I20260809 16:32:12.134294 19452 heartbeater.cc:347] Connected to a master server at 127.30.23.190:43727
I20260809 16:32:12.134408 19452 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:12.134613 19452 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
I20260809 16:32:12.135063 18657 ts_manager.cc:194] Registered new tserver with Master: 06dfe6fe35fe436db3eaaadedfc49984 (127.30.23.131:35917)
I20260809 16:32:12.135542 18657 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:49249
I20260809 16:32:12.138607 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:32:12.139213 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 18783
I20260809 16:32:12.175465 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 19010
I20260809 16:32:12.190236 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 19232
I20260809 16:32:12.206399 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 18611
I20260809 16:32:12.224804 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:43727
--webserver_interface=127.30.23.190
--webserver_port=44563
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:34087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:43727
--ranger_config_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.23.148:50931/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.23.148:50931/kms with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:12.308597 19456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:12.308807 19456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:12.308830 19456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:12.309934 19456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:32:12.309979 19456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:12.309993 19456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:12.310005 19456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:32:12.310017 19456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:32:12.311672 19456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:34087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.23.148:50931/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:43727
--ranger_config_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:43727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=44563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:12.311866 19456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:12.312038 19456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:12.317122 19470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:12.317123 19467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:12.317149 19464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:12.317142 19456 server_base.cc:1034] running on GCE node
I20260809 16:32:12.317652 19456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:12.317962 19456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:12.319139 19456 hybrid_clock.cc:648] HybridClock initialized: now 1786293132319122 us; error 51 us; skew 500 ppm
Aug 09 16:32:12 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:12.321911 19456 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:32:12.322383 19456 webserver.cc:460] Webserver started at http://127.30.23.190:44563/ using document root <none> and password file <none>
I20260809 16:32:12.322638 19456 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:12.322691 19456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:32:12 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:32:12.395573 19456 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:32:12.396397 19485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:12.396557 19456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:12.396649 19456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "6edcc300cfe542feb7353e2d9d6b067a"
format_stamp: "Formatted at 2026-08-09 16:32:10 on dist-test-slave-j2vl"
server_key: "ee6b466080acb8406883e7f48b68148a"
server_key_iv: "47752b0961bd201dfa2aa600b9293226"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:12.397173 19456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:12.410778 19456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:12.412827 19456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:12.412986 19456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:12.418041 19456 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:43727
I20260809 16:32:12.418128 19575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:43727 every 8 connection(s)
I20260809 16:32:12.418536 19456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:32:12.420231 19577 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:32:12.421119 19577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a: Bootstrap starting.
I20260809 16:32:12.421875 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 19456
I20260809 16:32:12.422219 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.23.148:50931/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:42205
--local_ip_for_outbound_sockets=127.30.23.129
--tserver_master_addrs=127.30.23.190:43727
--webserver_port=34211
--webserver_interface=127.30.23.129
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:34087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.23.148:50931/kms with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:32:12.432044 19577 log.cc:824] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:12.434980 19577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:32:12.435410 19577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a: Bootstrap complete.
I20260809 16:32:12.439410 19577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6edcc300cfe542feb7353e2d9d6b067a" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43727 } }
I20260809 16:32:12.439813 19577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6edcc300cfe542feb7353e2d9d6b067a, State: Initialized, Role: FOLLOWER
I20260809 16:32:12.440052 19577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6edcc300cfe542feb7353e2d9d6b067a" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43727 } }
I20260809 16:32:12.440128 19577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:32:12.440172 19577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:32:12.440263 19577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:32:12.441848 19577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6edcc300cfe542feb7353e2d9d6b067a" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43727 } }
I20260809 16:32:12.441982 19577 leader_election.cc:302] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6edcc300cfe542feb7353e2d9d6b067a; no voters:
I20260809 16:32:12.442206 19577 leader_election.cc:288] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:32:12.442292 19583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:32:12.442544 19583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [term 2 LEADER]: Becoming Leader. State: Replica: 6edcc300cfe542feb7353e2d9d6b067a, State: Running, Role: LEADER
I20260809 16:32:12.442585 19577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:32:12.443424 19583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6edcc300cfe542feb7353e2d9d6b067a" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43727 } }
I20260809 16:32:12.444114 19586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6edcc300cfe542feb7353e2d9d6b067a. Latest consensus state: current_term: 2 leader_uuid: "6edcc300cfe542feb7353e2d9d6b067a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6edcc300cfe542feb7353e2d9d6b067a" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43727 } } }
I20260809 16:32:12.444193 19586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:12.444309 19577 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 16:32:12.444415 19585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6edcc300cfe542feb7353e2d9d6b067a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6edcc300cfe542feb7353e2d9d6b067a" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43727 } } }
I20260809 16:32:12.444468 19585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:12.444834 19608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:32:12.448405 19608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:32:12.448910 19608 catalog_manager.cc:1259] Loaded cluster ID: 77fdf41737fc42e3bafae858a3e2980c
I20260809 16:32:12.449029 19608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:32:12.451639 19608 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:32:12.451987 19608 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a: Loaded TSK: 0
I20260809 16:32:12.452356 19608 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:32:12.528084 19581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:12.528328 19581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:12.528369 19581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:12.529699 19581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:12.529773 19581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:12.529820 19581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:32:12.531466 19581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:34087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.23.148:50931/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:42205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=34211
--tserver_master_addrs=127.30.23.190:43727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:12.531757 19581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:12.532081 19581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:12.539606 19649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:12.539610 19653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:12.540190 19647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:12.541781 19581 server_base.cc:1034] running on GCE node
I20260809 16:32:12.542027 19581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:12.542356 19581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:12.543583 19581 hybrid_clock.cc:648] HybridClock initialized: now 1786293132543566 us; error 29 us; skew 500 ppm
Aug 09 16:32:12 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:12.546586 19581 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:32:12.547822 19581 webserver.cc:460] Webserver started at http://127.30.23.129:34211/ using document root <none> and password file <none>
I20260809 16:32:12.548116 19581 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:12.548209 19581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:32:12 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:32:12.584295 19581 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:32:12.585493 19670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:12.585714 19581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:32:12.585832 19581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "504da40249894d4dacb0348b99731cf1"
format_stamp: "Formatted at 2026-08-09 16:32:11 on dist-test-slave-j2vl"
server_key: "de5d1536a71a46e9b47ae733f40666d2"
server_key_iv: "6242128ebf532053056d6350e5ec0b43"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:12.587065 19581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:12.603287 19581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:12.605989 19581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:12.606240 19581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:12.606532 19581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:12.607020 19581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:12.607110 19581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:12.607209 19581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:12.607280 19581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:12.616711 19581 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:42205
I20260809 16:32:12.617995 19581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:32:12.621265 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 19581
I20260809 16:32:12.621678 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.23.148:50931/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:34685
--local_ip_for_outbound_sockets=127.30.23.130
--tserver_master_addrs=127.30.23.190:43727
--webserver_port=43149
--webserver_interface=127.30.23.130
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:34087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.23.148:50931/kms with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:32:12.622656 19875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:42205 every 8 connection(s)
Aug 09 16:32:12 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:12.635778 19878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:12.619820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:43727 (local address 127.30.23.129:35613)
0809 16:32:12.620467 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:12.620515 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:12.621156 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:12.623389 (+ 2233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:12.623394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:12.623640 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:12.623943 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:12.623950 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:12.625741 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:12.625743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:12.627022 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:12.627026 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:12.628029 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:12.628030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:12.628162 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:12.628843 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:12.628855 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:12.630616 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:12.633408 (+ 2792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:12.633419 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:12.633422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:12.633767 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:12.634265 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:12.634266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:12.634266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:12.634445 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:12.635026 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:12.635039 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:12.635212 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 16:32:12.635326 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 16:32:12.635528 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":101,"threads_started":1}
I20260809 16:32:12.636900 19880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:12.620224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:35613 (local address 127.30.23.190:43727)
0809 16:32:12.622862 (+ 2638us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:12.622873 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:12.622919 (+ 46us) server_negotiation.cc:407] Connection header received
0809 16:32:12.623081 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:12.623088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:12.623150 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:12.623314 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:12.624094 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:12.625603 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:12.627138 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:12.627768 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:12.627903 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:12.630809 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:12.630815 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:12.630815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:12.630831 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:12.633237 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:12.634010 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:12.634015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:12.634016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:12.634164 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:12.634530 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:12.634531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:12.634531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:12.634874 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:12.634978 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:12.636520 (+ 1542us) server_negotiation.cc:299] Negotiation successful
0809 16:32:12.636658 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2567,"thread_start_us":94,"threads_started":1}
I20260809 16:32:12.638028 19876 heartbeater.cc:347] Connected to a master server at 127.30.23.190:43727
I20260809 16:32:12.638127 19876 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:12.638319 19876 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
W20260809 16:32:12.639389 19876 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:43727 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 16:32:12.639467 19876 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:12.639533 19876 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
I20260809 16:32:12.639948 19876 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:12.640027 19876 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
W20260809 16:32:12.640376 19876 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:32:12.782797 19881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:12.783178 19881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:12.783286 19881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:12.785133 19881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:12.785429 19881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:12.785547 19881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:32:12.787902 19881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:34087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.23.148:50931/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:34685
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=43149
--tserver_master_addrs=127.30.23.190:43727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:12.788439 19881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:12.788779 19881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:12.798713 19890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:12.799106 19889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:12.799342 19903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:12.800770 19881 server_base.cc:1034] running on GCE node
I20260809 16:32:12.801021 19881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:12.801386 19881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:12.802541 19881 hybrid_clock.cc:648] HybridClock initialized: now 1786293132802520 us; error 42 us; skew 500 ppm
Aug 09 16:32:12 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:12.812233 19881 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:32:12.815061 19881 webserver.cc:460] Webserver started at http://127.30.23.130:43149/ using document root <none> and password file <none>
I20260809 16:32:12.815471 19881 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:12.815596 19881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:32:12 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:32:12.858028 19881 fs_manager.cc:709] Time spent opening directory manager: real 0.019s user 0.000s sys 0.001s
I20260809 16:32:12.867415 19918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:12.867659 19881 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.000s sys 0.000s
I20260809 16:32:12.867782 19881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "a1061ab3a87c40999929bcb1280e73dd"
format_stamp: "Formatted at 2026-08-09 16:32:11 on dist-test-slave-j2vl"
server_key: "b47d85dd8df96b1092ac338243bbcd23"
server_key_iv: "5fb1da30913485e1e676b82cb9ca9f3f"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:12.868290 19881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:12.912286 19881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:12.915078 19881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:12.915241 19881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:12.915514 19881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:12.915863 19881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:12.915910 19881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:12.915948 19881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:12.915966 19881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:12.922493 19881 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:34685
I20260809 16:32:12.922958 19881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:32:12.926499 20100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:34685 every 8 connection(s)
I20260809 16:32:12.932510 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 19881
I20260809 16:32:12.932832 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.23.148:50931/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:35917
--local_ip_for_outbound_sockets=127.30.23.131
--tserver_master_addrs=127.30.23.190:43727
--webserver_port=33681
--webserver_interface=127.30.23.131
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:34087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.23.148:50931/kms with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:32:12 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:12.942994 19880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:12.927632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:39533 (local address 127.30.23.190:43727)
0809 16:32:12.927685 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:12.927687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:12.927955 (+ 268us) server_negotiation.cc:407] Connection header received
0809 16:32:12.928744 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:12.928746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:12.928764 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:12.928802 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:12.929836 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:12.930635 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:12.932395 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:12.932741 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:12.932804 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:12.938059 (+ 5255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:12.938061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:12.938061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:12.938073 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:12.940226 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:12.941452 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:12.941454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:12.941454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:12.941532 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:12.942023 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:12.942025 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:12.942025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:12.942204 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:12.942273 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:12.942754 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 16:32:12.942883 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:32:12.952986 20106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:12.927327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:43727 (local address 127.30.23.130:39533)
0809 16:32:12.927763 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:12.927832 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:12.928583 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:12.929041 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:12.929046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:12.929333 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:12.929707 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:12.929715 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:12.930755 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:12.930757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:12.932254 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:12.932258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:12.933099 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:12.933100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:12.933282 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:12.934057 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:12.934070 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:12.937508 (+ 3438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:12.940886 (+ 3378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:12.940897 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:12.940901 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:12.941296 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:12.941635 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:12.941636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:12.941637 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:12.941913 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:12.942323 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:12.942330 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:12.942567 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 16:32:12.949055 (+ 6488us) client_negotiation.cc:240] Negotiation successful
0809 16:32:12.950092 (+ 1037us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":95,"threads_started":1}
I20260809 16:32:12.953627 20102 heartbeater.cc:347] Connected to a master server at 127.30.23.190:43727
I20260809 16:32:12.953755 20102 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:12.954110 20102 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
W20260809 16:32:12.954988 20102 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:43727 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 16:32:12.955090 20102 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:12.955183 20102 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
I20260809 16:32:12.955569 20102 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:12.955669 20102 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
W20260809 16:32:12.956032 20102 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:32:13.063143 20107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:13.063530 20107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:13.063611 20107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:13.065752 20107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:13.065878 20107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:13.065963 20107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:32:13.068186 20107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:34087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.23.148:50931/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:35917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=33681
--tserver_master_addrs=127.30.23.190:43727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:13.068674 20107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:13.068993 20107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:13.075923 20117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:13.075941 20115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:13.076308 20124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:13.076727 20107 server_base.cc:1034] running on GCE node
I20260809 16:32:13.077046 20107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:13.077311 20107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:13.078514 20107 hybrid_clock.cc:648] HybridClock initialized: now 1786293133078478 us; error 54 us; skew 500 ppm
Aug 09 16:32:13 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:13.082422 20107 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:32:13.083108 20107 webserver.cc:460] Webserver started at http://127.30.23.131:33681/ using document root <none> and password file <none>
I20260809 16:32:13.083487 20107 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:13.083631 20107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:32:13 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:32:13.111019 20107 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.001s sys 0.000s
I20260809 16:32:13.112367 20134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:13.112589 20107 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:13.112676 20107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "06dfe6fe35fe436db3eaaadedfc49984"
format_stamp: "Formatted at 2026-08-09 16:32:12 on dist-test-slave-j2vl"
server_key: "2a7a6394dbed4e7bea5a2b65f9d08614"
server_key_iv: "2ad9a8e7c8019a26e6ba0f723999233b"
server_key_version: "kuduclusterkey@0"
I20260809 16:32:13.113193 20107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:13.148808 20107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:13.152278 20107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:13.152442 20107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:13.152694 20107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:13.153064 20107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:13.153110 20107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:13.153137 20107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:13.153154 20107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:13.160882 20107 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:35917
I20260809 16:32:13.160974 20331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:35917 every 8 connection(s)
I20260809 16:32:13.161681 20107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:32:13.165659 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 20107
Aug 09 16:32:13 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:13.173445 19880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:13.162632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:58025 (local address 127.30.23.190:43727)
0809 16:32:13.162710 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:13.162713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:13.163051 (+ 338us) server_negotiation.cc:407] Connection header received
0809 16:32:13.163711 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:13.163713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:13.163727 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:13.163798 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:13.164540 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:13.165289 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:13.166799 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:13.167074 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:13.167120 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:13.169812 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:13.169814 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:13.169814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:13.169824 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:13.171670 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:13.172202 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:13.172204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:13.172204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:13.172275 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:13.172586 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:13.172588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:13.172588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:13.172728 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:13.172786 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:13.173208 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 16:32:13.173308 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 16:32:13.173964 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:13.162626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:43727 (local address 127.30.23.131:58025)
0809 16:32:13.162904 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:13.162950 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:13.163571 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:13.163882 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:13.163886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:13.164120 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:13.164402 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:13.164409 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:13.165443 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:13.165443 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:13.166683 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:13.166686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:13.167237 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:13.167239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:13.167380 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:13.167956 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:13.167967 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:13.169691 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:13.171780 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:13.171787 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:13.171790 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:13.172099 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:13.172355 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:13.172356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:13.172356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:13.172505 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:13.172868 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:13.172875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:13.173044 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 16:32:13.173553 (+ 509us) client_negotiation.cc:240] Negotiation successful
0809 16:32:13.173724 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":89,"threads_started":1}
I20260809 16:32:13.174481 20333 heartbeater.cc:347] Connected to a master server at 127.30.23.190:43727
I20260809 16:32:13.174664 20333 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:13.174929 20333 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
I20260809 16:32:13.181623 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:13.167355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55802 (local address 127.30.23.190:43727)
0809 16:32:13.172211 (+ 4856us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:13.172213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:13.172221 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:32:13.172240 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:13.172241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:13.172255 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:13.172350 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:13.175221 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:13.175834 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:13.176812 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:13.177035 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:13.177117 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:13.178677 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:13.178679 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:13.178679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:13.178686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:13.180134 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:13.180543 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:13.180544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:13.180545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:13.180612 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:13.180859 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:13.180861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:13.180861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:13.180983 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:13.181176 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:13.181213 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 16:32:13.181317 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4831,"thread_start_us":43,"threads_started":1}
W20260809 16:32:13.185856 20333 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:43727 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 16:32:13.186038 20333 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:13.186161 20333 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
I20260809 16:32:13.186838 20333 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:13.186954 20333 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
W20260809 16:32:13.187319 20333 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:32:13.640635 19876 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:13.640844 19876 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
I20260809 16:32:13.641366 19511 ts_manager.cc:194] Registered new tserver with Master: 504da40249894d4dacb0348b99731cf1 (127.30.23.129:42205)
I20260809 16:32:13.642472 19511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:35613
Aug 09 16:32:13 dist-test-slave-j2vl krb5kdc[17909](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786293133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:13.956269 20102 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:13.956466 20102 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
I20260809 16:32:13.956844 19511 ts_manager.cc:194] Registered new tserver with Master: a1061ab3a87c40999929bcb1280e73dd (127.30.23.130:34685)
I20260809 16:32:13.957258 19511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:39533
I20260809 16:32:14.187525 20333 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:14.187693 20333 heartbeater.cc:511] Master 127.30.23.190:43727 requested a full tablet report, sending...
I20260809 16:32:14.188004 19511 ts_manager.cc:194] Registered new tserver with Master: 06dfe6fe35fe436db3eaaadedfc49984 (127.30.23.131:35917)
I20260809 16:32:14.188555 19511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:58025
I20260809 16:32:14.196614 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:32:14 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:32:14.220530 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:14.201651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55816 (local address 127.30.23.190:43727)
0809 16:32:14.206517 (+ 4866us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:14.206520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:14.206734 (+ 214us) server_negotiation.cc:407] Connection header received
0809 16:32:14.206763 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:14.206764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:14.206777 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:14.206837 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:14.207577 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.208938 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.214266 (+ 5328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.214761 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.214861 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:14.216780 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:14.216782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:14.216782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:14.216792 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:14.218882 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:14.219357 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:14.219358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:14.219359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:14.219424 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:14.219628 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:14.219629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:14.219629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:14.219773 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:14.219876 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:14.220322 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 16:32:14.220421 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4832,"thread_start_us":79,"threads_started":1}
I20260809 16:32:14.222925 19511 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:32:14.225220 19511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:32:14.250414 20390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:14.242756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:35917 (local address 127.0.0.1:59594)
0809 16:32:14.243855 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:14.243912 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:14.244004 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:14.244629 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:14.244634 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:14.244637 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:14.244725 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:14.244727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.246824 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.246826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:14.248466 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:14.248471 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.250027 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.250028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:14.250140 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:14.250157 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 16:32:14.250233 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 16:32:14.250300 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"thread_start_us":129,"threads_started":1}
I20260809 16:32:14.253823 20388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:14.242756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:34685 (local address 127.0.0.1:38650)
0809 16:32:14.249121 (+ 6365us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:14.249134 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:14.249198 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:14.249602 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:14.249603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:14.249606 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:14.249670 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:14.249672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.250827 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.250828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:14.251935 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:14.251939 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.253555 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.253556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:14.253627 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:14.253631 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:32:14.253688 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:32:14.253731 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6336,"mutex_wait_us":3,"thread_start_us":127,"threads_started":1}
I20260809 16:32:14.254177 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:14.243486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.30.23.130:34685)
0809 16:32:14.244147 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:14.244153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:14.249273 (+ 5120us) server_negotiation.cc:407] Connection header received
0809 16:32:14.249323 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:14.249327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:14.249400 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:14.249487 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:14.249749 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.250721 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.252037 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.253084 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.253167 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:14.253245 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:14.254040 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 16:32:14.254090 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":65,"threads_started":1}
I20260809 16:32:14.254949 19991 tablet_service.cc:1467] Processing CreateTablet for tablet 9f051a3f3a71444eb0f3128a0312ea8e (DEFAULT_TABLE table=test-table [id=8f05a07968664ed0a71d36231c7c4793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:32:14.255390 19991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f051a3f3a71444eb0f3128a0312ea8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:14.259289 20394 tablet_bootstrap.cc:492] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd: Bootstrap starting.
I20260809 16:32:14.260143 20394 tablet_bootstrap.cc:654] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:14.260553 20394 log.cc:824] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:14.268379 20390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:14.243063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:42205 (local address 127.0.0.1:58194)
0809 16:32:14.250913 (+ 7850us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:14.250925 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:14.250969 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:14.251298 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:14.251299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:14.251301 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:14.251337 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:14.251338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.252739 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.252740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:14.254464 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:14.254468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.268038 (+ 13570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.268041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:14.268143 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:14.268149 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:32:14.268206 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:32:14.268270 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7834,"thread_start_us":84,"threads_started":1}
I20260809 16:32:14.268926 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:14.243660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58194 (local address 127.30.23.129:42205)
0809 16:32:14.244304 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:14.244308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:14.251044 (+ 6736us) server_negotiation.cc:407] Connection header received
0809 16:32:14.251094 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:14.251097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:14.251116 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:14.251194 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:14.251432 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.252632 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.254578 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.255806 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.267835 (+ 12029us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:14.267978 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:14.268732 (+ 754us) server_negotiation.cc:299] Negotiation successful
0809 16:32:14.268841 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":61,"threads_started":1}
I20260809 16:32:14.269403 20391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:14.243304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59594 (local address 127.30.23.131:35917)
0809 16:32:14.244347 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:14.244351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:14.244362 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:32:14.244401 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:14.244403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:14.244421 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:14.244510 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:14.244819 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.246707 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.248829 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.249821 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.249914 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:14.249984 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:14.269197 (+ 19213us) server_negotiation.cc:299] Negotiation successful
0809 16:32:14.269265 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":888,"thread_start_us":58,"threads_started":1}
I20260809 16:32:14.270536 19748 tablet_service.cc:1467] Processing CreateTablet for tablet 9f051a3f3a71444eb0f3128a0312ea8e (DEFAULT_TABLE table=test-table [id=8f05a07968664ed0a71d36231c7c4793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:32:14.270972 19748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f051a3f3a71444eb0f3128a0312ea8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:14.274253 20394 tablet_bootstrap.cc:492] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd: No bootstrap required, opened a new log
I20260809 16:32:14.274358 20394 ts_tablet_manager.cc:1397] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260809 16:32:14.276757 20189 tablet_service.cc:1467] Processing CreateTablet for tablet 9f051a3f3a71444eb0f3128a0312ea8e (DEFAULT_TABLE table=test-table [id=8f05a07968664ed0a71d36231c7c4793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:32:14.277207 20189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f051a3f3a71444eb0f3128a0312ea8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:14.277469 20394 raft_consensus.cc:348] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dfe6fe35fe436db3eaaadedfc49984" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35917 } } peers { permanent_uuid: "a1061ab3a87c40999929bcb1280e73dd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34685 } } peers { permanent_uuid: "504da40249894d4dacb0348b99731cf1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 42205 } }
I20260809 16:32:14.277658 20394 raft_consensus.cc:374] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:14.277694 20394 raft_consensus.cc:729] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1061ab3a87c40999929bcb1280e73dd, State: Initialized, Role: FOLLOWER
I20260809 16:32:14.277854 20394 consensus_queue.cc:260] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dfe6fe35fe436db3eaaadedfc49984" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35917 } } peers { permanent_uuid: "a1061ab3a87c40999929bcb1280e73dd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34685 } } peers { permanent_uuid: "504da40249894d4dacb0348b99731cf1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 42205 } }
I20260809 16:32:14.278156 20394 ts_tablet_manager.cc:1428] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 16:32:14.278530 20102 heartbeater.cc:503] Master 127.30.23.190:43727 was elected leader, sending a full tablet report...
I20260809 16:32:14.283703 20398 tablet_bootstrap.cc:492] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1: Bootstrap starting.
I20260809 16:32:14.284632 20398 tablet_bootstrap.cc:654] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:14.284893 20401 tablet_bootstrap.cc:492] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984: Bootstrap starting.
I20260809 16:32:14.285673 20401 tablet_bootstrap.cc:654] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:14.286136 20398 log.cc:824] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:14.286563 20401 log.cc:824] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:14.288015 20398 tablet_bootstrap.cc:492] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1: No bootstrap required, opened a new log
I20260809 16:32:14.288151 20398 ts_tablet_manager.cc:1397] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 16:32:14.291246 20398 raft_consensus.cc:348] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dfe6fe35fe436db3eaaadedfc49984" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35917 } } peers { permanent_uuid: "a1061ab3a87c40999929bcb1280e73dd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34685 } } peers { permanent_uuid: "504da40249894d4dacb0348b99731cf1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 42205 } }
I20260809 16:32:14.291617 20398 raft_consensus.cc:374] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:14.291709 20398 raft_consensus.cc:729] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 504da40249894d4dacb0348b99731cf1, State: Initialized, Role: FOLLOWER
I20260809 16:32:14.291950 20398 consensus_queue.cc:260] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dfe6fe35fe436db3eaaadedfc49984" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35917 } } peers { permanent_uuid: "a1061ab3a87c40999929bcb1280e73dd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34685 } } peers { permanent_uuid: "504da40249894d4dacb0348b99731cf1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 42205 } }
I20260809 16:32:14.292562 20398 ts_tablet_manager.cc:1428] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 16:32:14.292816 19876 heartbeater.cc:503] Master 127.30.23.190:43727 was elected leader, sending a full tablet report...
I20260809 16:32:14.294240 20401 tablet_bootstrap.cc:492] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984: No bootstrap required, opened a new log
I20260809 16:32:14.294395 20401 ts_tablet_manager.cc:1397] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260809 16:32:14.297688 20401 raft_consensus.cc:348] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dfe6fe35fe436db3eaaadedfc49984" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35917 } } peers { permanent_uuid: "a1061ab3a87c40999929bcb1280e73dd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34685 } } peers { permanent_uuid: "504da40249894d4dacb0348b99731cf1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 42205 } }
I20260809 16:32:14.298007 20401 raft_consensus.cc:374] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:14.298086 20401 raft_consensus.cc:729] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06dfe6fe35fe436db3eaaadedfc49984, State: Initialized, Role: FOLLOWER
I20260809 16:32:14.298307 20401 consensus_queue.cc:260] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dfe6fe35fe436db3eaaadedfc49984" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35917 } } peers { permanent_uuid: "a1061ab3a87c40999929bcb1280e73dd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34685 } } peers { permanent_uuid: "504da40249894d4dacb0348b99731cf1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 42205 } }
I20260809 16:32:14.298722 20401 ts_tablet_manager.cc:1428] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 16:32:14.298893 20333 heartbeater.cc:503] Master 127.30.23.190:43727 was elected leader, sending a full tablet report...
W20260809 16:32:14.370528 19877 tablet.cc:2369] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:32:14.423470 20334 tablet.cc:2369] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:32:14.428375 20103 tablet.cc:2369] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:32:14.452253 20407 raft_consensus.cc:482] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:32:14.452438 20407 raft_consensus.cc:504] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dfe6fe35fe436db3eaaadedfc49984" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35917 } } peers { permanent_uuid: "a1061ab3a87c40999929bcb1280e73dd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34685 } } peers { permanent_uuid: "504da40249894d4dacb0348b99731cf1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 42205 } }
I20260809 16:32:14.452908 20407 leader_election.cc:288] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1061ab3a87c40999929bcb1280e73dd (127.30.23.130:34685), 504da40249894d4dacb0348b99731cf1 (127.30.23.129:42205)
I20260809 16:32:14.457616 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:14.453471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:51411 (local address 127.30.23.130:34685)
0809 16:32:14.453517 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:14.453520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:14.454019 (+ 499us) server_negotiation.cc:407] Connection header received
0809 16:32:14.454134 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:14.454135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:14.454146 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:14.454178 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:14.454418 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.455151 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.456569 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.457159 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.457209 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:14.457219 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:14.457483 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 16:32:14.457532 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:32:14.457906 20408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:14.453208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:34685 (local address 127.30.23.131:51411)
0809 16:32:14.453933 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:14.453957 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:14.454051 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:14.454251 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:14.454251 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:14.454253 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:14.454340 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:14.454342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.455271 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.455272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:14.456471 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:14.456474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.457301 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.457301 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:14.457392 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:14.457396 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:32:14.457764 (+ 368us) client_negotiation.cc:240] Negotiation successful
0809 16:32:14.457807 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":328,"threads_started":1}
I20260809 16:32:14.458377 20027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f051a3f3a71444eb0f3128a0312ea8e" candidate_uuid: "06dfe6fe35fe436db3eaaadedfc49984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1061ab3a87c40999929bcb1280e73dd" is_pre_election: true
I20260809 16:32:14.458606 20027 raft_consensus.cc:2454] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06dfe6fe35fe436db3eaaadedfc49984 in term 0.
I20260809 16:32:14.458902 20141 leader_election.cc:302] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06dfe6fe35fe436db3eaaadedfc49984, a1061ab3a87c40999929bcb1280e73dd; no voters:
I20260809 16:32:14.459050 20407 raft_consensus.cc:2781] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:32:14.459096 20407 raft_consensus.cc:482] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:32:14.459122 20407 raft_consensus.cc:3037] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:14.460160 20407 raft_consensus.cc:504] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dfe6fe35fe436db3eaaadedfc49984" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35917 } } peers { permanent_uuid: "a1061ab3a87c40999929bcb1280e73dd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34685 } } peers { permanent_uuid: "504da40249894d4dacb0348b99731cf1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 42205 } }
I20260809 16:32:14.460325 20407 leader_election.cc:288] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [CANDIDATE]: Term 1 election: Requested vote from peers a1061ab3a87c40999929bcb1280e73dd (127.30.23.130:34685), 504da40249894d4dacb0348b99731cf1 (127.30.23.129:42205)
I20260809 16:32:14.460659 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:14.453850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:48461 (local address 127.30.23.129:42205)
0809 16:32:14.453911 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:14.453913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:14.456319 (+ 2406us) server_negotiation.cc:407] Connection header received
0809 16:32:14.456423 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:14.456424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:14.456435 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:14.456481 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:14.456695 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.457391 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.459502 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.460132 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.460188 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:14.460199 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:14.460526 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 16:32:14.460585 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:32:14.460659 20027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f051a3f3a71444eb0f3128a0312ea8e" candidate_uuid: "06dfe6fe35fe436db3eaaadedfc49984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1061ab3a87c40999929bcb1280e73dd"
I20260809 16:32:14.460749 20027 raft_consensus.cc:3037] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:14.460939 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:14.453664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:42205 (local address 127.30.23.131:48461)
0809 16:32:14.456238 (+ 2574us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:14.456251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:14.456349 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:14.456554 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:14.456555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:14.456556 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:14.456617 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:14.456618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.457494 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.457495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:14.459384 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:14.459387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.460265 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.460265 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:14.460336 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:14.460342 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:32:14.460790 (+ 448us) client_negotiation.cc:240] Negotiation successful
0809 16:32:14.460840 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2553,"thread_start_us":81,"threads_started":1}
I20260809 16:32:14.461760 20027 raft_consensus.cc:2454] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06dfe6fe35fe436db3eaaadedfc49984 in term 1.
I20260809 16:32:14.461989 20141 leader_election.cc:302] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06dfe6fe35fe436db3eaaadedfc49984, a1061ab3a87c40999929bcb1280e73dd; no voters:
I20260809 16:32:14.462177 20407 raft_consensus.cc:2781] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:32:14.462354 20407 raft_consensus.cc:686] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [term 1 LEADER]: Becoming Leader. State: Replica: 06dfe6fe35fe436db3eaaadedfc49984, State: Running, Role: LEADER
I20260809 16:32:14.462471 20407 consensus_queue.cc:237] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dfe6fe35fe436db3eaaadedfc49984" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35917 } } peers { permanent_uuid: "a1061ab3a87c40999929bcb1280e73dd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34685 } } peers { permanent_uuid: "504da40249894d4dacb0348b99731cf1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 42205 } }
I20260809 16:32:14.463058 19787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f051a3f3a71444eb0f3128a0312ea8e" candidate_uuid: "06dfe6fe35fe436db3eaaadedfc49984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "504da40249894d4dacb0348b99731cf1" is_pre_election: true
I20260809 16:32:14.463220 19787 raft_consensus.cc:2454] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06dfe6fe35fe436db3eaaadedfc49984 in term 0.
I20260809 16:32:14.463819 19511 catalog_manager.cc:5495] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06dfe6fe35fe436db3eaaadedfc49984 (127.30.23.131). New cstate: current_term: 1 leader_uuid: "06dfe6fe35fe436db3eaaadedfc49984" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06dfe6fe35fe436db3eaaadedfc49984" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1061ab3a87c40999929bcb1280e73dd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "504da40249894d4dacb0348b99731cf1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 42205 } health_report { overall_health: UNKNOWN } } }
I20260809 16:32:14.473433 19787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f051a3f3a71444eb0f3128a0312ea8e" candidate_uuid: "06dfe6fe35fe436db3eaaadedfc49984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "504da40249894d4dacb0348b99731cf1"
I20260809 16:32:14.473587 19787 raft_consensus.cc:3037] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:14.474684 19787 raft_consensus.cc:2454] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06dfe6fe35fe436db3eaaadedfc49984 in term 1.
I20260809 16:32:14.483525 20391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:14.476185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59598 (local address 127.30.23.131:35917)
0809 16:32:14.479755 (+ 3570us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:14.479758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:14.479773 (+ 15us) server_negotiation.cc:407] Connection header received
0809 16:32:14.479824 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:14.479825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:14.479867 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:14.479969 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:32:14.480211 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.481083 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.482374 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:14.482659 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:14.482722 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:14.482939 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:32:14.483083 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:32:14.483152 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:14.483414 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 16:32:14.483439 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3531}
I20260809 16:32:14.487648 19787 raft_consensus.cc:1261] T 9f051a3f3a71444eb0f3128a0312ea8e P 504da40249894d4dacb0348b99731cf1 [term 1 FOLLOWER]: Refusing update from remote peer 06dfe6fe35fe436db3eaaadedfc49984: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:32:14.488049 20407 consensus_queue.cc:1035] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [LEADER]: Connected to new peer: Peer: permanent_uuid: "504da40249894d4dacb0348b99731cf1" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 42205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:32:14.490429 20027 raft_consensus.cc:1261] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd [term 1 FOLLOWER]: Refusing update from remote peer 06dfe6fe35fe436db3eaaadedfc49984: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:32:14.490764 20410 consensus_queue.cc:1035] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1061ab3a87c40999929bcb1280e73dd" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 34685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:32:14.494992 20413 mvcc.cc:205] Tried to move back new op lower bound from 7316656678857531392 to 7316656678758838272. Current Snapshot: MvccSnapshot[applied={T|T < 7316656678857531392}]
I20260809 16:32:14.497833 20418 mvcc.cc:205] Tried to move back new op lower bound from 7316656678857531392 to 7316656678758838272. Current Snapshot: MvccSnapshot[applied={T|T < 7316656678857531392}]
I20260809 16:32:14.498282 20421 mvcc.cc:205] Tried to move back new op lower bound from 7316656678857531392 to 7316656678758838272. Current Snapshot: MvccSnapshot[applied={T|T < 7316656678857531392}]
I20260809 16:32:14.498274 19511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:32:14.498374 19511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:32:14.499191 19511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6edcc300cfe542feb7353e2d9d6b067a: Sending DeleteTablet for 3 replicas of tablet 9f051a3f3a71444eb0f3128a0312ea8e
I20260809 16:32:14.499574 20189 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f051a3f3a71444eb0f3128a0312ea8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:32:14 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:59594
I20260809 16:32:14.500154 19991 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f051a3f3a71444eb0f3128a0312ea8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:32:14 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:38650
I20260809 16:32:14.500545 19748 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f051a3f3a71444eb0f3128a0312ea8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:32:14 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:58194
I20260809 16:32:14.501439 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 19581
I20260809 16:32:14.517820 20434 tablet_replica.cc:330] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd: stopping tablet replica
I20260809 16:32:14.518006 20434 raft_consensus.cc:2229] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:32:14.518174 20434 raft_consensus.cc:2258] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:32:14.518397 20432 tablet_replica.cc:330] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984: stopping tablet replica
I20260809 16:32:14.518528 20432 raft_consensus.cc:2229] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:32:14.518646 20432 raft_consensus.cc:2258] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:32:14.524711 20434 ts_tablet_manager.cc:1905] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:32:14.527103 20434 ts_tablet_manager.cc:1918] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:32:14.527213 20434 log.cc:1196] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/9f051a3f3a71444eb0f3128a0312ea8e
I20260809 16:32:14.527541 20434 ts_tablet_manager.cc:1939] T 9f051a3f3a71444eb0f3128a0312ea8e P a1061ab3a87c40999929bcb1280e73dd: Deleting consensus metadata
I20260809 16:32:14.528249 20432 ts_tablet_manager.cc:1905] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:32:14.529790 19489 catalog_manager.cc:4841] TS a1061ab3a87c40999929bcb1280e73dd (127.30.23.130:34685): tablet 9f051a3f3a71444eb0f3128a0312ea8e (table test-table [id=8f05a07968664ed0a71d36231c7c4793]) successfully deleted
I20260809 16:32:14.530778 20432 ts_tablet_manager.cc:1918] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:32:14.530886 20432 log.cc:1196] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/9f051a3f3a71444eb0f3128a0312ea8e
I20260809 16:32:14.531194 20432 ts_tablet_manager.cc:1939] T 9f051a3f3a71444eb0f3128a0312ea8e P 06dfe6fe35fe436db3eaaadedfc49984: Deleting consensus metadata
I20260809 16:32:14.531906 19487 catalog_manager.cc:4841] TS 06dfe6fe35fe436db3eaaadedfc49984 (127.30.23.131:35917): tablet 9f051a3f3a71444eb0f3128a0312ea8e (table test-table [id=8f05a07968664ed0a71d36231c7c4793]) successfully deleted
W20260809 16:32:14.534014 19489 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:42205 (error 108)
I20260809 16:32:14.535172 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 19881
I20260809 16:32:14.575434 20390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:14.575279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:42205 (local address 127.0.0.1:58196)
0809 16:32:14.575336 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:14.575387 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:42205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 16:32:14.576548 19489 catalog_manager.cc:4584] TS 504da40249894d4dacb0348b99731cf1 (127.30.23.129:42205): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9f051a3f3a71444eb0f3128a0312ea8e: Network error: Client connection negotiation failed: client connection to 127.30.23.129:42205: connect: Connection refused (error 111)
Aug 09 16:32:14 dist-test-slave-j2vl krb5kdc[17909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for HTTP/127.30.23.148@KRBTEST.COM
I20260809 16:32:14.586413 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 20107
I20260809 16:32:14.610005 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 19456
I20260809 16:32:14.625818 30814 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 16:32:14.977043 30814 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 16:32:14.977152 30814 mini_ranger.cc:66] Stopping Ranger...
I20260809 16:32:15.323159 30814 mini_ranger.cc:68] Stopped Ranger
2026-08-09 16:32:15.323 UTC [17961] LOG: received smart shutdown request
2026-08-09 16:32:15.328 UTC [17961] LOG: background worker "logical replication launcher" (PID 17973) exited with exit code 1
2026-08-09 16:32:15.328 UTC [17966] LOG: shutting down
2026-08-09 16:32:15.534 UTC [17961] LOG: database system is shut down
2026-08-09T16:32:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54611 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:32:15 dist-test-slave-j2vl krb5kdc[20452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:32:15 dist-test-slave-j2vl krb5kdc[20452](info): set up 2 sockets
Aug 09 16:32:15 dist-test-slave-j2vl krb5kdc[20452](info): commencing operation
krb5kdc: starting...
W20260809 16:32:16.948838 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:32:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:32:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:32:17.098932 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:44725
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:44725,127.30.23.189:40495,127.30.23.188:38647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:17.188309 20472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:17.188587 20472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:17.188629 20472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:17.189889 20472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:32:17.189967 20472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:17.189991 20472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:17.190013 20472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:32:17.190042 20472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:32:17.191942 20472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:44725,127.30.23.189:40495,127.30.23.188:38647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:44725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:17.192277 20472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:17.192559 20472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:32:17.198328 20472 server_base.cc:1034] running on GCE node
W20260809 16:32:17.198293 20482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:17.198320 20483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:17.198293 20487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:17.199167 20472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:17.199533 20472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:17.200727 20472 hybrid_clock.cc:648] HybridClock initialized: now 1786293137200703 us; error 40 us; skew 500 ppm
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:17.203630 20472 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:32:17.204214 20472 webserver.cc:460] Webserver started at http://127.30.23.190:33445/ using document root <none> and password file <none>
I20260809 16:32:17.204478 20472 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:17.204550 20472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:17.204704 20472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:32:17.205965 20472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "cc929088764b4c62b50268a1841fb441"
format_stamp: "Formatted at 2026-08-09 16:32:17 on dist-test-slave-j2vl"
server_key: "d42a080b8aecb7a033dd8479ce021d46"
server_key_iv: "6d07be26daa71fe798b1229ca8e70283"
server_key_version: "encryptionkey@0"
I20260809 16:32:17.206344 20472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "cc929088764b4c62b50268a1841fb441"
format_stamp: "Formatted at 2026-08-09 16:32:17 on dist-test-slave-j2vl"
server_key: "d42a080b8aecb7a033dd8479ce021d46"
server_key_iv: "6d07be26daa71fe798b1229ca8e70283"
server_key_version: "encryptionkey@0"
I20260809 16:32:17.207755 20472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:32:17.208669 20496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:17.208963 20472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:17.209100 20472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "cc929088764b4c62b50268a1841fb441"
format_stamp: "Formatted at 2026-08-09 16:32:17 on dist-test-slave-j2vl"
server_key: "d42a080b8aecb7a033dd8479ce021d46"
server_key_iv: "6d07be26daa71fe798b1229ca8e70283"
server_key_version: "encryptionkey@0"
I20260809 16:32:17.209188 20472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:17.240798 20472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:17.243515 20472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:17.243753 20472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:17.249131 20472 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:44725
I20260809 16:32:17.249279 20574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:44725 every 8 connection(s)
I20260809 16:32:17.249629 20472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:32:17.250571 20576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:17.252276 20576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:32:17.255998 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 20472
I20260809 16:32:17.256137 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:32:17.256480 30814 external_mini_cluster.cc:1428] Setting key fe002221a0c69d8a19f7ae53e428376c
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:17.263960 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.253064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44725 (local address 127.0.0.1:50516)
0809 16:32:17.253249 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.253286 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:17.253853 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:17.254198 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:17.254201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:17.254491 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:17.254666 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.254670 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.255787 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.255788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.256617 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.256620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.257464 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.257465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.257534 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.258289 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:17.258299 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:17.259782 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:17.262330 (+ 2548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.262337 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.262340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.262560 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.262826 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.262827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.262827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.262918 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.263322 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:17.263327 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:17.263400 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 16:32:17.263505 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 16:32:17.263766 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":96,"thread_start_us":73,"threads_started":1}
I20260809 16:32:17.263958 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.253229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50516 (local address 127.30.23.190:44725)
0809 16:32:17.253398 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.253402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.253432 (+ 30us) server_negotiation.cc:407] Connection header received
0809 16:32:17.254024 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.254026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.254044 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.254130 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.254822 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.255680 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.256714 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.257232 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.257392 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.259886 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.259894 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.259895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.259903 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.262192 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.262654 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.262657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.262658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.262754 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.262999 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.263000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.263000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.263236 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.263345 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.263539 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.263690 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260809 16:32:17.264912 20576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:32:17.265573 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.265289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:40495 (local address 127.0.0.1:32880)
0809 16:32:17.265474 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.265544 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260809 16:32:17.266052 20499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.23.189:40495: connect: Connection refused (error 111)
I20260809 16:32:17.266924 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.266795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:40495 (local address 127.0.0.1:32888)
0809 16:32:17.266866 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.266901 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 16:32:17.267093 20576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.189:40495: Network error: Client connection negotiation failed: client connection to 127.30.23.189:40495: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.30.23.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.189.keytab.
Entry for principal kudu/127.30.23.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.189.keytab.
I20260809 16:32:17.294076 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.189:40495
--webserver_interface=127.30.23.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.189.keytab
--principal=kudu/127.30.23.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:44725,127.30.23.189:40495,127.30.23.188:38647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:32:17.316428 20576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } attempt: 1
I20260809 16:32:17.317168 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.316996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:40495 (local address 127.0.0.1:32894)
0809 16:32:17.317093 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.317133 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 16:32:17.317680 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.317551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:40495 (local address 127.0.0.1:32910)
0809 16:32:17.317620 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.317655 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 16:32:17.317842 20576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.189:40495: Network error: Client connection negotiation failed: client connection to 127.30.23.189:40495: connect: Connection refused (error 111)
I20260809 16:32:17.386037 20576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } attempt: 2
I20260809 16:32:17.386546 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.386397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:40495 (local address 127.0.0.1:32912)
0809 16:32:17.386471 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.386515 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 16:32:17.386940 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.386836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:40495 (local address 127.0.0.1:32914)
0809 16:32:17.386886 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.386919 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 16:32:17.387060 20576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.189:40495: Network error: Client connection negotiation failed: client connection to 127.30.23.189:40495: connect: Connection refused (error 111)
W20260809 16:32:17.389202 20589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:17.389467 20589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:17.389518 20589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:17.390766 20589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:32:17.390839 20589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:17.390868 20589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:17.390902 20589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:32:17.390926 20589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:32:17.392460 20589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:44725,127.30.23.189:40495,127.30.23.188:38647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.189.keytab
--principal=kudu/127.30.23.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.189:40495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.23.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:17.392747 20589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:17.392987 20589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:17.398185 20599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:17.398190 20600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:17.398327 20589 server_base.cc:1034] running on GCE node
W20260809 16:32:17.398239 20602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:17.398743 20589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:17.399091 20589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:17.400287 20589 hybrid_clock.cc:648] HybridClock initialized: now 1786293137400276 us; error 48 us; skew 500 ppm
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:17.403014 20589 init.cc:377] Logged in from keytab as kudu/127.30.23.189@KRBTEST.COM (short username kudu)
I20260809 16:32:17.403527 20589 webserver.cc:460] Webserver started at http://127.30.23.189:39005/ using document root <none> and password file <none>
I20260809 16:32:17.403772 20589 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:17.403844 20589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:17.404065 20589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:32:17.405177 20589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/data/instance:
uuid: "a2262f982373411a9152a2b050afe816"
format_stamp: "Formatted at 2026-08-09 16:32:17 on dist-test-slave-j2vl"
server_key: "1b426c5d6cfc74d2d0d56fd84aae44d4"
server_key_iv: "6fd6f63ab08b4596ae8a0aa523e30ab5"
server_key_version: "encryptionkey@0"
I20260809 16:32:17.405594 20589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/wal/instance:
uuid: "a2262f982373411a9152a2b050afe816"
format_stamp: "Formatted at 2026-08-09 16:32:17 on dist-test-slave-j2vl"
server_key: "1b426c5d6cfc74d2d0d56fd84aae44d4"
server_key_iv: "6fd6f63ab08b4596ae8a0aa523e30ab5"
server_key_version: "encryptionkey@0"
I20260809 16:32:17.407133 20589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:17.407963 20612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:17.408181 20589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:17.408295 20589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/wal
uuid: "a2262f982373411a9152a2b050afe816"
format_stamp: "Formatted at 2026-08-09 16:32:17 on dist-test-slave-j2vl"
server_key: "1b426c5d6cfc74d2d0d56fd84aae44d4"
server_key_iv: "6fd6f63ab08b4596ae8a0aa523e30ab5"
server_key_version: "encryptionkey@0"
I20260809 16:32:17.408391 20589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:17.418154 20589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:17.420550 20589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:17.420737 20589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:17.425100 20589 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.189:40495
I20260809 16:32:17.425136 20700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.189:40495 every 8 connection(s)
I20260809 16:32:17.425590 20589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/data/info.pb
I20260809 16:32:17.426520 20701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:17.428125 20701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:32:17.431107 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 20589
I20260809 16:32:17.431222 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-1/wal/instance
I20260809 16:32:17.431438 30814 external_mini_cluster.cc:1428] Setting key 3168467746d65ef8faff45f260846efe
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.189@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:17.439035 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.428976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50518 (local address 127.30.23.190:44725)
0809 16:32:17.429049 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.429051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.429276 (+ 225us) server_negotiation.cc:407] Connection header received
0809 16:32:17.429795 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.429797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.429811 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.429878 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.430537 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.431206 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.432363 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.432697 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.432774 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.435376 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.435378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.435378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.435388 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.437474 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.437901 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.437902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.437903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.437981 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.438234 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.438235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.438236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.438433 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.438515 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.438795 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.438917 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 16:32:17.439064 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.428939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44725 (local address 127.0.0.1:50518)
0809 16:32:17.429115 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.429162 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:17.429684 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:17.429937 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:17.429940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:17.430226 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:17.430441 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.430445 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.431323 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.431324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.432272 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.432275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.432805 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.432806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.432934 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.433572 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:17.433582 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:17.435273 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:17.437567 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.437576 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.437578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.437813 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.438055 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.438055 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.438056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.438160 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.438511 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:17.438517 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:17.438647 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 16:32:17.438738 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 16:32:17.438877 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260809 16:32:17.439740 20701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.23.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.188@KRBTEST.COM" created.
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.189@KRBTEST.COM for kudu/127.30.23.189@KRBTEST.COM
I20260809 16:32:17.448354 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.440056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:40495 (local address 127.0.0.1:32918)
0809 16:32:17.440122 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.440132 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:17.440175 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:17.440481 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:17.440482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:17.440640 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:17.440689 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.440691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.441665 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.441666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.442155 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.442156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.442633 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.442634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.442701 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.443134 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:17.443138 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:17.445298 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:17.447260 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.447261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.447262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.447433 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.447664 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.447664 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.447665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.447734 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.448027 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:17.448028 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:17.448067 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 16:32:17.448129 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:32:17.448237 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 16:32:17.448441 20708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.440140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32918 (local address 127.30.23.189:40495)
0809 16:32:17.440336 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.440340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.440356 (+ 16us) server_negotiation.cc:407] Connection header received
0809 16:32:17.440372 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.440374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.440384 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.440479 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.440772 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.441555 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.442253 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.442548 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.442625 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.445489 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.445498 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.445498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.445506 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.447143 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.447531 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.447535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.447535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.447594 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.447799 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.447800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.447800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.447953 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.448051 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.448168 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.448313 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":170,"thread_start_us":127,"threads_started":1}
I20260809 16:32:17.449026 20701 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:32:17.449434 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.449282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46562)
0809 16:32:17.449331 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.449401 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 16:32:17.449565 20613 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
I20260809 16:32:17.450023 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.449919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46564)
0809 16:32:17.449972 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.449997 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 16:32:17.450137 20701 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:38647: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.188.keytab.
Entry for principal kudu/127.30.23.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.188.keytab.
I20260809 16:32:17.464704 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.188:38647
--webserver_interface=127.30.23.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.188.keytab
--principal=kudu/127.30.23.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:44725,127.30.23.189:40495,127.30.23.188:38647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:32:17.478222 20576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } attempt: 3
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for kudu/127.30.23.189@KRBTEST.COM
I20260809 16:32:17.486264 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.478666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:40495 (local address 127.0.0.1:32920)
0809 16:32:17.478739 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.478761 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:17.478819 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:17.478995 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:17.478996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:17.479183 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:17.479265 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.479268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.479978 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.479980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.480500 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.480502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.480906 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.480908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.480994 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.481551 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:17.481556 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:17.483073 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:17.485099 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.485100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.485101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.485287 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.485623 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.485624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.485624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.485694 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.485988 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:17.485989 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:17.486036 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 16:32:17.486095 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:32:17.486186 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 16:32:17.486326 20708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.478786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32920 (local address 127.30.23.189:40495)
0809 16:32:17.478861 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.478863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.478871 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:32:17.478901 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.478902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.478921 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.478986 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.479372 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.479887 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.480595 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.480824 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.480909 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.483170 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.483172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.483172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.483181 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.484995 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.485426 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.485428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.485429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.485500 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.485767 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.485768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.485768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.485917 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.485989 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.486138 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.486225 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 16:32:17.486824 20576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:32:17.487156 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.487059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46566)
0809 16:32:17.487097 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.487132 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 16:32:17.487531 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.487448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46572)
0809 16:32:17.487490 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.487515 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 16:32:17.487646 20576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:38647: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
I20260809 16:32:17.499399 20701 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } attempt: 1
I20260809 16:32:17.499881 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.499746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46588)
0809 16:32:17.499811 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.499849 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 16:32:17.500353 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.500225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46592)
0809 16:32:17.500295 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.500329 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 16:32:17.500492 20701 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:38647: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
I20260809 16:32:17.518812 20576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } attempt: 1
I20260809 16:32:17.519357 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.519205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46598)
0809 16:32:17.519276 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.519323 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 16:32:17.519814 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.519704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46606)
0809 16:32:17.519759 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.519790 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 16:32:17.520061 20576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:38647: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
W20260809 16:32:17.553893 20711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:17.554122 20711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:17.554147 20711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:17.555366 20711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:32:17.555415 20711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:17.555430 20711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:17.555444 20711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:32:17.555455 20711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:32:17.557049 20711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:44725,127.30.23.189:40495,127.30.23.188:38647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.188.keytab
--principal=kudu/127.30.23.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.188:38647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.23.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:17.557322 20711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:17.557600 20711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:32:17.563098 20711 server_base.cc:1034] running on GCE node
W20260809 16:32:17.563081 20727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:17.563081 20721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:17.563068 20720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:17.563565 20711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:17.563895 20711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:17.565078 20711 hybrid_clock.cc:648] HybridClock initialized: now 1786293137565057 us; error 43 us; skew 500 ppm
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:17.568061 20711 init.cc:377] Logged in from keytab as kudu/127.30.23.188@KRBTEST.COM (short username kudu)
I20260809 16:32:17.568631 20711 webserver.cc:460] Webserver started at http://127.30.23.188:42073/ using document root <none> and password file <none>
I20260809 16:32:17.568696 20701 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } attempt: 2
I20260809 16:32:17.568898 20711 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:17.568971 20711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:17.569128 20711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:32:17.569170 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.569028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46616)
0809 16:32:17.569099 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.569124 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 16:32:17.569545 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.569445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46624)
0809 16:32:17.569494 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.569521 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 16:32:17.569635 20701 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:38647: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
I20260809 16:32:17.570468 20711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/data/instance:
uuid: "437c18f0d89648e0b4f1086dac362625"
format_stamp: "Formatted at 2026-08-09 16:32:17 on dist-test-slave-j2vl"
server_key: "0c37f977937594da3869bf863359591c"
server_key_iv: "b44543290c566c22ab4889b2ed167e35"
server_key_version: "encryptionkey@0"
I20260809 16:32:17.570860 20711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/wal/instance:
uuid: "437c18f0d89648e0b4f1086dac362625"
format_stamp: "Formatted at 2026-08-09 16:32:17 on dist-test-slave-j2vl"
server_key: "0c37f977937594da3869bf863359591c"
server_key_iv: "b44543290c566c22ab4889b2ed167e35"
server_key_version: "encryptionkey@0"
I20260809 16:32:17.572299 20711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:32:17.573138 20735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:17.573355 20711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:17.573470 20711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/wal
uuid: "437c18f0d89648e0b4f1086dac362625"
format_stamp: "Formatted at 2026-08-09 16:32:17 on dist-test-slave-j2vl"
server_key: "0c37f977937594da3869bf863359591c"
server_key_iv: "b44543290c566c22ab4889b2ed167e35"
server_key_version: "encryptionkey@0"
I20260809 16:32:17.573529 20711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:17.594810 20711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:17.595273 20576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } attempt: 2
I20260809 16:32:17.595822 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.595691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46640)
0809 16:32:17.595752 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.595791 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 16:32:17.596235 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.596123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46652)
0809 16:32:17.596179 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.596206 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 16:32:17.596352 20576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:38647: Network error: Client connection negotiation failed: client connection to 127.30.23.188:38647: connect: Connection refused (error 111)
I20260809 16:32:17.597432 20711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:17.597623 20711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:17.602250 20711 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.188:38647
I20260809 16:32:17.602283 20823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.188:38647 every 8 connection(s)
I20260809 16:32:17.602780 20711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/data/info.pb
I20260809 16:32:17.603463 20824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:17.605160 20824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.188@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:17.610970 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 20711
I20260809 16:32:17.611150 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786292963539133-30814-0/minicluster-data/master-2/wal/instance
I20260809 16:32:17.611419 30814 external_mini_cluster.cc:1428] Setting key 261dd35db95fbef0124395ac19737336
I20260809 16:32:17.615154 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.605973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50520 (local address 127.30.23.190:44725)
0809 16:32:17.606038 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.606040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.606236 (+ 196us) server_negotiation.cc:407] Connection header received
0809 16:32:17.606675 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.606676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.606687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.606736 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.607446 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.607901 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.608834 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.609029 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.609082 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.612087 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.612088 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.612089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.612097 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.613701 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.614148 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.614148 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.614149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.614238 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.614494 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.614495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.614495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.614639 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.614726 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.614963 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.615057 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:32:17.615295 20826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.605890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:44725 (local address 127.0.0.1:50520)
0809 16:32:17.606113 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.606149 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:17.606592 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:17.606786 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:17.606789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:17.607145 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:17.607374 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.607378 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.607977 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.607977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.608758 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.608761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.609118 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.609119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.609232 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.609797 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:17.609806 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:17.611756 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:17.613776 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.613783 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.613785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.614072 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.614305 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.614306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.614306 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.614423 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.614718 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:17.614722 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:17.614838 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 16:32:17.614921 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 16:32:17.615067 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":86,"threads_started":1}
I20260809 16:32:17.615837 20824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.188@KRBTEST.COM
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.188@KRBTEST.COM for kudu/127.30.23.189@KRBTEST.COM
I20260809 16:32:17.621459 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.611870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.30.23.188:38647)
0809 16:32:17.612052 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.612056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.612097 (+ 41us) server_negotiation.cc:407] Connection header received
0809 16:32:17.612122 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.612124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.612136 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.612238 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.613074 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.614105 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.615082 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.615532 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.615599 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.618309 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.618319 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.618319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.618327 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.620154 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.620544 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.620547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.620548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.620615 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.620831 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.620832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.620833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.620997 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.621174 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.621228 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.621310 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
I20260809 16:32:17.623428 20826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.616110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:40495 (local address 127.0.0.1:32936)
0809 16:32:17.616214 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.616224 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:17.616273 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:17.616430 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:17.616431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:17.616544 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:17.616588 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.616589 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.617551 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.617552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.618115 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.618118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.618671 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.618672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.618747 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.619034 (+ 287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:17.619039 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:17.620458 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:17.622390 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.622391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.622392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.622559 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.622787 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.622788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.622788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.622855 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.623118 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:17.623119 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:17.623161 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 16:32:17.623235 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:32:17.623331 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 16:32:17.623448 20708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.616170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.30.23.189:40495)
0809 16:32:17.616214 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.616216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.616259 (+ 43us) server_negotiation.cc:407] Connection header received
0809 16:32:17.616341 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.616342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.616353 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.616411 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.616668 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.617453 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.618206 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.618542 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.618611 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.620561 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.620563 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.620563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.620571 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.622294 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.622646 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.622647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.622648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.622710 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.622923 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.622924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.622925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.623042 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.623114 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.623277 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.623361 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:32:17.624023 20824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.188@KRBTEST.COM for kudu/127.30.23.188@KRBTEST.COM
I20260809 16:32:17.630080 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.622330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50524 (local address 127.30.23.190:44725)
0809 16:32:17.622377 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.622378 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.622396 (+ 18us) server_negotiation.cc:407] Connection header received
0809 16:32:17.622454 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.622455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.622464 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.622520 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.623000 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.623497 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.624442 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.624646 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.624731 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.627033 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.627035 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.627035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.627044 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.628585 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.629011 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.629012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.629013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.629075 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.629329 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.629331 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.629331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.629553 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.629801 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.629849 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.629949 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:32:17.630980 20826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.624292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46666)
0809 16:32:17.624341 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.624350 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:17.624387 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:17.624499 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:17.624500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:17.624632 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:17.624657 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.624659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.625276 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.625277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.625782 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.625784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.626217 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.626218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.626266 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.626665 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:17.626670 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:17.627953 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:17.629677 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.629678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.629679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.629950 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.630159 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.630160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.630161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.630240 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.630580 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:17.630580 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:17.630642 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 16:32:17.630719 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:32:17.630830 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 16:32:17.631091 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.624339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46666 (local address 127.30.23.188:38647)
0809 16:32:17.624374 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.624376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.624384 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:32:17.624441 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.624442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.624451 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.624499 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.624738 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.625200 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.625877 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.626135 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.626207 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.628043 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.628044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.628044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.628050 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.629573 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.630034 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.630035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.630035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.630090 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.630337 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.630338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.630339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.630512 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.630588 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.630760 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.630901 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":19}
I20260809 16:32:17.632818 20824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625: Bootstrap starting.
I20260809 16:32:17.634227 20824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:17.634575 20824 log.cc:824] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:17.635439 20824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625: No bootstrap required, opened a new log
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.189@KRBTEST.COM
I20260809 16:32:17.637882 20824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } }
I20260809 16:32:17.638082 20824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:17.638129 20824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437c18f0d89648e0b4f1086dac362625, State: Initialized, Role: FOLLOWER
I20260809 16:32:17.638283 20824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } }
I20260809 16:32:17.638586 20831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } } }
I20260809 16:32:17.638648 20824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:32:17.638670 20831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:32:17.639634 20708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.630835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32940 (local address 127.30.23.189:40495)
0809 16:32:17.630909 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.630911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.630919 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:32:17.630946 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.630947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.630957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.631021 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.631771 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.632543 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.633272 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.633739 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.633830 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.636155 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.636157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.636157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.636167 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.638002 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.638526 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.638528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.638529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.638593 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.638893 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.638894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.638895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.639061 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.639347 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.639417 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.639521 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 16:32:17.641155 20846 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:32:17.641199 20846 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:32:17.641554 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:32:17.641088 20711 master_runner.cc:186] Error getting master registration for 127.30.23.190:44725: OK, instance_id { permanent_uuid: "cc929088764b4c62b50268a1841fb441" instance_seqno: 1786293137247662 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:32:17.642844 20711 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:32:17.654551 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.643350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.30.23.190:44725)
0809 16:32:17.643406 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.643409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.646084 (+ 2675us) server_negotiation.cc:407] Connection header received
0809 16:32:17.646180 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.646181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.646195 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.646231 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.647222 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.648035 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.649386 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.649688 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.649727 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.651384 (+ 1657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.651386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.651386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.651394 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.653068 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.653594 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.653596 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.653596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.653659 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.653883 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.653884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.653884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.654020 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.654108 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.654358 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.654446 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:32:17.654551 20708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.645901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32950 (local address 127.30.23.189:40495)
0809 16:32:17.645951 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.645953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.646182 (+ 229us) server_negotiation.cc:407] Connection header received
0809 16:32:17.646291 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.646292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.646304 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.646339 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.647222 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.648040 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.649903 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.650202 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.650247 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.651637 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.651639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.651639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.651647 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.653182 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.653625 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.653626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.653627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.653682 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.653896 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.653896 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.653897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.654020 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.654108 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.654358 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.654445 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:32:17.655516 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.643443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46668 (local address 127.30.23.188:38647)
0809 16:32:17.643486 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.643489 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.645594 (+ 2105us) server_negotiation.cc:407] Connection header received
0809 16:32:17.645679 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.645681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.645695 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.645759 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.646568 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.647320 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.648446 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.648925 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.649021 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.651509 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.651510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.651511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.651519 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.654124 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.654547 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.654549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.654549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.654609 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.654859 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.654861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.654861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.655022 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.655189 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.655327 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.655424 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:32:17.660787 20701 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } attempt: 3
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.189@KRBTEST.COM for kudu/127.30.23.188@KRBTEST.COM
I20260809 16:32:17.670732 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.661174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46684 (local address 127.30.23.188:38647)
0809 16:32:17.661212 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.661214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.661224 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:32:17.661305 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.661306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.661314 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.661439 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.661782 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.662245 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.663330 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.663614 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.664267 (+ 653us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.667785 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.667786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.667786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.667794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.669484 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.669999 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.670000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.670000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.670058 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.670268 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.670268 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.670269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.670379 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.670432 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.670603 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.670667 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:32:17.670822 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.661101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46684)
0809 16:32:17.661172 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.661186 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:17.661241 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:17.661447 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:17.661448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:17.661644 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:17.661695 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.661697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.662334 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.662336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.663228 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.663230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.663915 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.663916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.664019 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.665326 (+ 1307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:17.665333 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:17.667675 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:17.669652 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.669654 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.669654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.669924 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.670127 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.670128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.670129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.670202 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.670447 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:17.670448 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:17.670500 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 16:32:17.670587 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:32:17.670697 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 16:32:17.672283 20701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816: Bootstrap starting.
I20260809 16:32:17.672940 20701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:17.673242 20701 log.cc:824] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:17.674038 20701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816: No bootstrap required, opened a new log
I20260809 16:32:17.675956 20701 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } }
I20260809 16:32:17.676086 20701 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:17.676136 20701 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2262f982373411a9152a2b050afe816, State: Initialized, Role: FOLLOWER
I20260809 16:32:17.676306 20701 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } }
I20260809 16:32:17.676612 20860 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } } }
I20260809 16:32:17.676680 20701 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:32:17.676738 20860 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:32:17.678815 20878 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:32:17.678908 20878 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:32:17.678848 20589 master_runner.cc:186] Error getting master registration for 127.30.23.190:44725: OK, instance_id { permanent_uuid: "cc929088764b4c62b50268a1841fb441" instance_seqno: 1786293137247662 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:32:17.681566 20589 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:32:17.695508 20576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } attempt: 3
Aug 09 16:32:17 dist-test-slave-j2vl krb5kdc[20452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for kudu/127.30.23.188@KRBTEST.COM
I20260809 16:32:17.704282 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.695859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:38647 (local address 127.0.0.1:46700)
0809 16:32:17.695915 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:17.695928 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:17.695968 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:17.696112 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:17.696113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:17.696334 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:17.696391 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.696393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.697364 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.697365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.698190 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:17.698193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.698692 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.698693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:17.698750 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.699277 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:17.699283 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:17.700876 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:17.703167 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.703168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.703169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.703412 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.703619 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:17.703619 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:17.703620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:17.703672 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:17.703946 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:17.703947 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:17.703991 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 16:32:17.704062 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 16:32:17.704161 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:32:17.704303 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:17.695926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46700 (local address 127.30.23.188:38647)
0809 16:32:17.695977 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:17.695979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:17.695994 (+ 15us) server_negotiation.cc:407] Connection header received
0809 16:32:17.696022 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:17.696023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:17.696032 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:17.696087 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:17.696482 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.697244 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.698293 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:17.698608 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:17.698685 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:17.700970 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:17.700971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:17.700972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:17.700979 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:17.703081 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.703500 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.703501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.703501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.703557 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:17.703729 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:17.703730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:17.703730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:17.703859 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:17.703920 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:17.704119 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 16:32:17.704205 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:32:17.706262 20576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441: Bootstrap starting.
I20260809 16:32:17.707026 20576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:17.707351 20576 log.cc:824] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:17.708348 20576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441: No bootstrap required, opened a new log
I20260809 16:32:17.710454 20576 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } }
I20260809 16:32:17.710604 20576 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:17.710641 20576 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc929088764b4c62b50268a1841fb441, State: Initialized, Role: FOLLOWER
I20260809 16:32:17.710769 20576 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } }
I20260809 16:32:17.711057 20883 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } } }
I20260809 16:32:17.711215 20883 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:32:17.711126 20576 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 16:32:17.713495 20900 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:32:17.713608 20900 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:32:17.715860 20472 master_runner.cc:263] No leader master found from master cc929088764b4c62b50268a1841fb441
I20260809 16:32:17.715921 20472 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 16:32:17.744724 20503 tablet.cc:2369] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:32:17.840785 20883 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:32:17.840968 20883 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } }
I20260809 16:32:17.841370 20883 leader_election.cc:288] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2262f982373411a9152a2b050afe816 (127.30.23.189:40495), 437c18f0d89648e0b4f1086dac362625 (127.30.23.188:38647)
I20260809 16:32:17.841638 20653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc929088764b4c62b50268a1841fb441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2262f982373411a9152a2b050afe816" is_pre_election: true
I20260809 16:32:17.841676 20776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc929088764b4c62b50268a1841fb441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437c18f0d89648e0b4f1086dac362625" is_pre_election: true
I20260809 16:32:17.841832 20776 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc929088764b4c62b50268a1841fb441 in term 0.
I20260809 16:32:17.841845 20653 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc929088764b4c62b50268a1841fb441 in term 0.
I20260809 16:32:17.842087 20497 leader_election.cc:302] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437c18f0d89648e0b4f1086dac362625, cc929088764b4c62b50268a1841fb441; no voters:
I20260809 16:32:17.842236 20883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:32:17.842298 20883 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:32:17.842325 20883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:17.843178 20883 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } }
I20260809 16:32:17.843338 20883 leader_election.cc:288] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [CANDIDATE]: Term 1 election: Requested vote from peers a2262f982373411a9152a2b050afe816 (127.30.23.189:40495), 437c18f0d89648e0b4f1086dac362625 (127.30.23.188:38647)
I20260809 16:32:17.843549 20776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc929088764b4c62b50268a1841fb441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437c18f0d89648e0b4f1086dac362625"
I20260809 16:32:17.843617 20653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc929088764b4c62b50268a1841fb441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2262f982373411a9152a2b050afe816"
I20260809 16:32:17.843647 20776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:17.843694 20653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:17.844607 20776 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc929088764b4c62b50268a1841fb441 in term 1.
I20260809 16:32:17.844607 20653 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc929088764b4c62b50268a1841fb441 in term 1.
I20260809 16:32:17.844825 20497 leader_election.cc:302] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437c18f0d89648e0b4f1086dac362625, cc929088764b4c62b50268a1841fb441; no voters:
I20260809 16:32:17.844903 20883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:32:17.845078 20883 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [term 1 LEADER]: Becoming Leader. State: Replica: cc929088764b4c62b50268a1841fb441, State: Running, Role: LEADER
I20260809 16:32:17.845182 20883 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } }
I20260809 16:32:17.845693 20902 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc929088764b4c62b50268a1841fb441. Latest consensus state: current_term: 1 leader_uuid: "cc929088764b4c62b50268a1841fb441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } } }
I20260809 16:32:17.845796 20902 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:17.845999 20905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:32:17.846200 20905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:32:17.848110 20653 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [term 1 FOLLOWER]: Refusing update from remote peer cc929088764b4c62b50268a1841fb441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:32:17.848121 20776 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [term 1 FOLLOWER]: Refusing update from remote peer cc929088764b4c62b50268a1841fb441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 16:32:17.848325 20745 tablet.cc:2369] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:32:17.848446 20902 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:32:17.848541 20883 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:32:17.849833 20860 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc929088764b4c62b50268a1841fb441. Latest consensus state: current_term: 1 leader_uuid: "cc929088764b4c62b50268a1841fb441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } } }
I20260809 16:32:17.849936 20860 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:32:17.849882 20831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc929088764b4c62b50268a1841fb441. Latest consensus state: current_term: 1 leader_uuid: "cc929088764b4c62b50268a1841fb441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } } }
I20260809 16:32:17.849982 20831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:32:17.851034 20902 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc929088764b4c62b50268a1841fb441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } } }
I20260809 16:32:17.851152 20902 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:17.851217 20908 mvcc.cc:205] Tried to move back new op lower bound from 7316656692623486976 to 7316656692614590464. Current Snapshot: MvccSnapshot[applied={T|T < 7316656692623486976}]
I20260809 16:32:17.851541 20883 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc929088764b4c62b50268a1841fb441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } } }
I20260809 16:32:17.851620 20883 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441 [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:17.851861 20860 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc929088764b4c62b50268a1841fb441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } } }
I20260809 16:32:17.851951 20860 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:32:17.852145 20831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc929088764b4c62b50268a1841fb441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc929088764b4c62b50268a1841fb441" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 44725 } } peers { permanent_uuid: "a2262f982373411a9152a2b050afe816" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 40495 } } peers { permanent_uuid: "437c18f0d89648e0b4f1086dac362625" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 38647 } } }
I20260809 16:32:17.852247 20831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:32:17.852414 20910 mvcc.cc:205] Tried to move back new op lower bound from 7316656692623486976 to 7316656692614590464. Current Snapshot: MvccSnapshot[applied={T|T < 7316656692623486976}]
I20260809 16:32:17.852564 20909 mvcc.cc:205] Tried to move back new op lower bound from 7316656692623486976 to 7316656692614590464. Current Snapshot: MvccSnapshot[applied={T|T < 7316656692623486976}]
I20260809 16:32:17.852874 20905 catalog_manager.cc:1347] Generated new cluster ID: 425ced3c26f946be84f02e80377fbae3
I20260809 16:32:17.852944 20905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:32:17.861083 20905 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:32:17.861745 20905 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:32:17.874178 20905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc929088764b4c62b50268a1841fb441: Generated new TSK 0
I20260809 16:32:17.874416 20905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:32:18.641757 20846 catalog_manager.cc:1259] Loaded cluster ID: 425ced3c26f946be84f02e80377fbae3
I20260809 16:32:18.641846 20846 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625: loading cluster ID for follower catalog manager: success
I20260809 16:32:18.642875 20846 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625: acquiring CA information for follower catalog manager: success
I20260809 16:32:18.643222 20846 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 437c18f0d89648e0b4f1086dac362625: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:32:18.679410 20878 catalog_manager.cc:1259] Loaded cluster ID: 425ced3c26f946be84f02e80377fbae3
I20260809 16:32:18.679494 20878 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816: loading cluster ID for follower catalog manager: success
I20260809 16:32:18.680339 20878 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816: acquiring CA information for follower catalog manager: success
I20260809 16:32:18.680613 20878 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a2262f982373411a9152a2b050afe816: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:32:18.936823 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 20472
I20260809 16:32:18.949482 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 20589
I20260809 16:32:18.963606 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 20711
2026-08-09T16:32:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3311 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:32:19 dist-test-slave-j2vl krb5kdc[20925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:32:19 dist-test-slave-j2vl krb5kdc[20925](info): set up 2 sockets
Aug 09 16:32:19 dist-test-slave-j2vl krb5kdc[20925](info): commencing operation
krb5kdc: starting...
W20260809 16:32:20.243338 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:32:20 dist-test-slave-j2vl krb5kdc[20925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:32:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:32:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:32:20.373325 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:35353
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:35353
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:20.457202 20955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:20.457489 20955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:20.457530 20955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:20.458791 20955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:32:20.458856 20955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:20.458880 20955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:20.458906 20955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:32:20.458937 20955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:32:20.460443 20955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:35353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:35353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:20.460738 20955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:20.460978 20955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:20.467942 20965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:20.468017 20968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:20.468106 20963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:20.468788 20955 server_base.cc:1034] running on GCE node
I20260809 16:32:20.469084 20955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:20.469465 20955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:20.470634 20955 hybrid_clock.cc:648] HybridClock initialized: now 1786293140470611 us; error 42 us; skew 500 ppm
Aug 09 16:32:20 dist-test-slave-j2vl krb5kdc[20925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293140, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:20.473426 20955 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:32:20.473907 20955 webserver.cc:460] Webserver started at http://127.30.23.190:33951/ using document root <none> and password file <none>
I20260809 16:32:20.474103 20955 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:20.474145 20955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:20.474237 20955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:32:20.475282 20955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "e262b81b0d034cc6ae1a3b516146ed95"
format_stamp: "Formatted at 2026-08-09 16:32:20 on dist-test-slave-j2vl"
server_key: "7b416421dfad224455b8542ea169c88c"
server_key_iv: "0aacf29b0a4e604870c5ae3224821151"
server_key_version: "encryptionkey@0"
I20260809 16:32:20.475587 20955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "e262b81b0d034cc6ae1a3b516146ed95"
format_stamp: "Formatted at 2026-08-09 16:32:20 on dist-test-slave-j2vl"
server_key: "7b416421dfad224455b8542ea169c88c"
server_key_iv: "0aacf29b0a4e604870c5ae3224821151"
server_key_version: "encryptionkey@0"
I20260809 16:32:20.476915 20955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:32:20.477698 20983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:20.477942 20955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:32:20.478050 20955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "e262b81b0d034cc6ae1a3b516146ed95"
format_stamp: "Formatted at 2026-08-09 16:32:20 on dist-test-slave-j2vl"
server_key: "7b416421dfad224455b8542ea169c88c"
server_key_iv: "0aacf29b0a4e604870c5ae3224821151"
server_key_version: "encryptionkey@0"
I20260809 16:32:20.478132 20955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:20.494854 20955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:20.497259 20955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:20.497449 20955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:20.501827 20955 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:35353
I20260809 16:32:20.501868 21057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:35353 every 8 connection(s)
I20260809 16:32:20.502259 20955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:32:20.503160 21059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:20.505602 21059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95: Bootstrap starting.
I20260809 16:32:20.506212 21059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:20.506515 21059 log.cc:824] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:20.507323 21059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95: No bootstrap required, opened a new log
I20260809 16:32:20.509017 21059 raft_consensus.cc:348] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e262b81b0d034cc6ae1a3b516146ed95" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35353 } }
I20260809 16:32:20.509200 21059 raft_consensus.cc:374] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:20.509248 21059 raft_consensus.cc:729] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e262b81b0d034cc6ae1a3b516146ed95, State: Initialized, Role: FOLLOWER
I20260809 16:32:20.509328 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 20955
I20260809 16:32:20.509459 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:32:20.509438 21059 consensus_queue.cc:260] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e262b81b0d034cc6ae1a3b516146ed95" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35353 } }
I20260809 16:32:20.509531 21059 raft_consensus.cc:388] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:32:20.509577 21059 raft_consensus.cc:482] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:32:20.509629 21059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:20.509671 30814 external_mini_cluster.cc:1428] Setting key 516b4e0bf587086e7f927e048b43e2a6
I20260809 16:32:20.510311 21059 raft_consensus.cc:504] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e262b81b0d034cc6ae1a3b516146ed95" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35353 } }
I20260809 16:32:20.510466 21059 leader_election.cc:302] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e262b81b0d034cc6ae1a3b516146ed95; no voters:
I20260809 16:32:20.510663 21059 leader_election.cc:288] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:32:20.510699 21064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:32:20.510879 21064 raft_consensus.cc:686] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [term 1 LEADER]: Becoming Leader. State: Replica: e262b81b0d034cc6ae1a3b516146ed95, State: Running, Role: LEADER
I20260809 16:32:20.510958 21059 sys_catalog.cc:564] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:32:20.510988 21064 consensus_queue.cc:237] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e262b81b0d034cc6ae1a3b516146ed95" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35353 } }
I20260809 16:32:20.511329 21064 sys_catalog.cc:455] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e262b81b0d034cc6ae1a3b516146ed95. Latest consensus state: current_term: 1 leader_uuid: "e262b81b0d034cc6ae1a3b516146ed95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e262b81b0d034cc6ae1a3b516146ed95" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35353 } } }
I20260809 16:32:20.511425 21064 sys_catalog.cc:458] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:20.511607 21070 sys_catalog.cc:455] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e262b81b0d034cc6ae1a3b516146ed95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e262b81b0d034cc6ae1a3b516146ed95" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35353 } } }
I20260809 16:32:20.511652 21078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:32:20.512323 21078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:32:20.512387 21070 sys_catalog.cc:458] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95 [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:20.514463 21078 catalog_manager.cc:1347] Generated new cluster ID: ee32d7b4d9d2410cbf44dd87ad35c689
I20260809 16:32:20.514520 21078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:32:20 dist-test-slave-j2vl krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:20.520576 21063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:20.510210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48004 (local address 127.30.23.190:35353)
0809 16:32:20.510413 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:20.510423 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:20.510465 (+ 42us) server_negotiation.cc:407] Connection header received
0809 16:32:20.510588 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:20.510596 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:20.510651 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:20.511716 (+ 1065us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:20.511859 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:20.513149 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:20.514224 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:20.514769 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:20.514896 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:20.517225 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:20.517229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:20.517229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:20.517241 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:20.519026 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:20.519399 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:20.519402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:20.519403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:20.519512 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:20.519732 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:20.519733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:20.519733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:20.519971 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:20.520152 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:20.520219 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 16:32:20.520375 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":103,"threads_started":1}
I20260809 16:32:20.521610 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:32:20.542080 21078 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:32:20.542649 21078 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:32:20.547538 21078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e262b81b0d034cc6ae1a3b516146ed95: Generated new TSK 0
I20260809 16:32:20.547716 21078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:32:20.575603 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 20955
2026-08-09T16:32:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1614 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:32:20 dist-test-slave-j2vl krb5kdc[21101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:32:20 dist-test-slave-j2vl krb5kdc[21101](info): set up 2 sockets
Aug 09 16:32:20 dist-test-slave-j2vl krb5kdc[21101](info): commencing operation
krb5kdc: starting...
W20260809 16:32:21.860739 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:32:21 dist-test-slave-j2vl krb5kdc[21101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:32:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:32:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:32:21.992501 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:38345
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:38345
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:22.077047 21121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:22.077296 21121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:22.077333 21121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:22.078486 21121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:32:22.078559 21121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:22.078585 21121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:22.078616 21121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:32:22.078642 21121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:32:22.080122 21121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:38345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:38345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:22.080404 21121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:22.080629 21121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:22.085695 21136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:22.085731 21129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:22.085832 21121 server_base.cc:1034] running on GCE node
W20260809 16:32:22.085695 21133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:22.086228 21121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:22.086516 21121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:22.087683 21121 hybrid_clock.cc:648] HybridClock initialized: now 1786293142087660 us; error 39 us; skew 500 ppm
Aug 09 16:32:22 dist-test-slave-j2vl krb5kdc[21101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293142, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:22.093318 21121 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:32:22.093861 21121 webserver.cc:460] Webserver started at http://127.30.23.190:40149/ using document root <none> and password file <none>
I20260809 16:32:22.094110 21121 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:22.094154 21121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:22.094323 21121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:32:22.095377 21121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "fc859467c0b846f78b48e10905f6db67"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "f45fce56ee31a0cd111a3dc5235a5d8f"
server_key_iv: "217d377993f3640954447347e052da9d"
server_key_version: "encryptionkey@0"
I20260809 16:32:22.095736 21121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "fc859467c0b846f78b48e10905f6db67"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "f45fce56ee31a0cd111a3dc5235a5d8f"
server_key_iv: "217d377993f3640954447347e052da9d"
server_key_version: "encryptionkey@0"
I20260809 16:32:22.097159 21121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 16:32:22.098030 21149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:22.098213 21121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:22.098320 21121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "fc859467c0b846f78b48e10905f6db67"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "f45fce56ee31a0cd111a3dc5235a5d8f"
server_key_iv: "217d377993f3640954447347e052da9d"
server_key_version: "encryptionkey@0"
I20260809 16:32:22.098408 21121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:22.115583 21121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:22.117559 21121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:22.117733 21121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:22.122200 21121 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:38345
I20260809 16:32:22.122254 21238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:38345 every 8 connection(s)
I20260809 16:32:22.122620 21121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:32:22.123651 21240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:22.126055 21240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Bootstrap starting.
I20260809 16:32:22.126662 21240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:22.126957 21240 log.cc:824] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:22.127681 21240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: No bootstrap required, opened a new log
I20260809 16:32:22.128021 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 21121
I20260809 16:32:22.128144 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:32:22.128330 30814 external_mini_cluster.cc:1428] Setting key de75e47cc41b8ae73b3017ef097077a5
I20260809 16:32:22.130208 21240 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } }
I20260809 16:32:22.130396 21240 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:22.130440 21240 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc859467c0b846f78b48e10905f6db67, State: Initialized, Role: FOLLOWER
I20260809 16:32:22.130578 21240 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } }
I20260809 16:32:22.130669 21240 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:32:22.130707 21240 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:32:22.130810 21240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:22.131388 21240 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } }
I20260809 16:32:22.131481 21240 leader_election.cc:302] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc859467c0b846f78b48e10905f6db67; no voters:
I20260809 16:32:22.131656 21240 leader_election.cc:288] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:32:22.131704 21246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:32:22.131860 21246 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 1 LEADER]: Becoming Leader. State: Replica: fc859467c0b846f78b48e10905f6db67, State: Running, Role: LEADER
I20260809 16:32:22.131955 21240 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:32:22.132000 21246 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } }
I20260809 16:32:22.132354 21246 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc859467c0b846f78b48e10905f6db67. Latest consensus state: current_term: 1 leader_uuid: "fc859467c0b846f78b48e10905f6db67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } } }
I20260809 16:32:22.132467 21246 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:22.132772 21248 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc859467c0b846f78b48e10905f6db67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } } }
I20260809 16:32:22.132853 21248 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:22.133448 21263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:32:22.134032 21263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:32:22.136118 21263 catalog_manager.cc:1347] Generated new cluster ID: 6545bd167ca74307ae9cbeea61b5e4ae
I20260809 16:32:22.136171 21263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:32:22 dist-test-slave-j2vl krb5kdc[21101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:22.141206 21245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.128878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60164 (local address 127.30.23.190:38345)
0809 16:32:22.129073 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:22.129080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:22.129105 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:32:22.129759 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:22.129768 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:22.129816 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:22.129950 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:22.130783 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.132479 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.134018 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.134573 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.134696 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.137625 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:22.137629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:22.137629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:22.137640 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:22.139553 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:22.139965 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:22.139968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:22.139968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:22.140081 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:22.140296 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:22.140297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:22.140298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:22.140621 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:22.140768 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:22.140898 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 16:32:22.141021 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
I20260809 16:32:22.147354 21263 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:32:22.148063 21263 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:32:22.157747 21263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Generated new TSK 0
I20260809 16:32:22.158020 21263 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:32:22.188395 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:38345
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:22.277839 21286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:22.278074 21286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:22.278111 21286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:22.279260 21286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:22.279340 21286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:22.279388 21286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:32:22.280874 21286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:38345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:22.281144 21286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:22.281418 21286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:22.287186 21298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:22.287186 21293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:22.287225 21294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:22.287524 21286 server_base.cc:1034] running on GCE node
I20260809 16:32:22.287740 21286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:22.287994 21286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:22.289155 21286 hybrid_clock.cc:648] HybridClock initialized: now 1786293142289132 us; error 39 us; skew 500 ppm
Aug 09 16:32:22 dist-test-slave-j2vl krb5kdc[21101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293142, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:22.291965 21286 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:32:22.292379 21286 webserver.cc:460] Webserver started at http://127.30.23.129:37311/ using document root <none> and password file <none>
I20260809 16:32:22.292619 21286 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:22.292666 21286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:22.292825 21286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:32:22.294004 21286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "0b8ed3a2424f40d9929e7f62be54ecbf"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "43bf4d71816d5c6186e23449ecd7e3de"
server_key_iv: "e8da4a38acc49f2aa391d8da782cc21f"
server_key_version: "encryptionkey@0"
I20260809 16:32:22.294453 21286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "0b8ed3a2424f40d9929e7f62be54ecbf"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "43bf4d71816d5c6186e23449ecd7e3de"
server_key_iv: "e8da4a38acc49f2aa391d8da782cc21f"
server_key_version: "encryptionkey@0"
I20260809 16:32:22.295879 21286 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:32:22.296792 21306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:22.297078 21286 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:22.297163 21286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "0b8ed3a2424f40d9929e7f62be54ecbf"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "43bf4d71816d5c6186e23449ecd7e3de"
server_key_iv: "e8da4a38acc49f2aa391d8da782cc21f"
server_key_version: "encryptionkey@0"
I20260809 16:32:22.297276 21286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:22.316599 21286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:22.319177 21286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:22.319342 21286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:22.319634 21286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:22.319988 21286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:22.320024 21286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:22.320046 21286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:22.320091 21286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:22.326224 21286 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:46363
I20260809 16:32:22.326323 21507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:46363 every 8 connection(s)
I20260809 16:32:22.326655 21286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:32:22 dist-test-slave-j2vl krb5kdc[21101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293142, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:22.334785 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 21286
I20260809 16:32:22.334916 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:32:22.335109 30814 external_mini_cluster.cc:1428] Setting key 6995675bab47764bacc81e63c6fdc9f4
I20260809 16:32:22.337126 21245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.327487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:48343 (local address 127.30.23.190:38345)
0809 16:32:22.327553 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:22.327555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:22.327800 (+ 245us) server_negotiation.cc:407] Connection header received
0809 16:32:22.328233 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:22.328234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:22.328246 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:22.328321 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:22.329318 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.329857 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.330914 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.331124 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.331191 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.333849 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:22.333851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:22.333852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:22.333860 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:22.335588 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:22.336118 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:22.336119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:22.336119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:22.336176 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:22.336484 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:22.336485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:22.336485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:22.336589 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:22.336643 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:22.336982 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 16:32:22.337061 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:32:22.337464 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.327429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:48343)
0809 16:32:22.327606 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:22.327650 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:22.328140 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:22.328395 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:22.328399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:22.329016 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:22.329212 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.329217 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.329964 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.329965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.330797 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.330800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.331233 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.331234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.331419 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.332021 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:22.332030 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:22.333735 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:22.335669 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:22.335676 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:22.335678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:22.336026 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:22.336243 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:22.336244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:22.336244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:22.336409 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:22.336656 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:22.336663 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:22.336840 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 16:32:22.336956 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 16:32:22.337145 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260809 16:32:22.338032 21508 heartbeater.cc:347] Connected to a master server at 127.30.23.190:38345
I20260809 16:32:22.338157 21508 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:22.338342 21508 heartbeater.cc:511] Master 127.30.23.190:38345 requested a full tablet report, sending...
I20260809 16:32:22.338915 21183 ts_manager.cc:194] Registered new tserver with Master: 0b8ed3a2424f40d9929e7f62be54ecbf (127.30.23.129:46363)
I20260809 16:32:22.339553 21183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:48343
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:32:22.364210 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:38345
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:22.446799 21516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:22.447001 21516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:22.447024 21516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:22.448089 21516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:22.448125 21516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:22.448150 21516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:32:22.449548 21516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:38345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:22.449741 21516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:22.449941 21516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:22.454986 21526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:22.455091 21529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:22.455195 21516 server_base.cc:1034] running on GCE node
W20260809 16:32:22.455003 21531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:22.455430 21516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:22.455652 21516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:22.456823 21516 hybrid_clock.cc:648] HybridClock initialized: now 1786293142456797 us; error 37 us; skew 500 ppm
Aug 09 16:32:22 dist-test-slave-j2vl krb5kdc[21101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293142, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:22.459429 21516 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:32:22.459949 21516 webserver.cc:460] Webserver started at http://127.30.23.130:42043/ using document root <none> and password file <none>
I20260809 16:32:22.460210 21516 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:22.460278 21516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:22.460439 21516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:32:22.461772 21516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "b275b0d7161a4602a97533b6445cf042"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "642fc3068f5ba762b088aa71bf0b43c8"
server_key_iv: "474022df76c511b08cfd5a6fe422abad"
server_key_version: "encryptionkey@0"
I20260809 16:32:22.462248 21516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "b275b0d7161a4602a97533b6445cf042"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "642fc3068f5ba762b088aa71bf0b43c8"
server_key_iv: "474022df76c511b08cfd5a6fe422abad"
server_key_version: "encryptionkey@0"
I20260809 16:32:22.464432 21516 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 16:32:22.466411 21546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:22.466656 21516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:32:22.466761 21516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "b275b0d7161a4602a97533b6445cf042"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "642fc3068f5ba762b088aa71bf0b43c8"
server_key_iv: "474022df76c511b08cfd5a6fe422abad"
server_key_version: "encryptionkey@0"
I20260809 16:32:22.466851 21516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:22.490761 21516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:22.492676 21516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:22.492792 21516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:22.493037 21516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:22.493398 21516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:22.493438 21516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:22.493467 21516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:22.493480 21516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:22.499619 21516 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:42163
I20260809 16:32:22.499663 21724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:42163 every 8 connection(s)
I20260809 16:32:22.499990 21516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:32:22 dist-test-slave-j2vl krb5kdc[21101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293142, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:22.509394 21245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.500799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:39891 (local address 127.30.23.190:38345)
0809 16:32:22.500869 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:22.500871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:22.501115 (+ 244us) server_negotiation.cc:407] Connection header received
0809 16:32:22.501691 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:22.501692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:22.501702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:22.501764 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:22.502726 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.503236 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.504234 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.504443 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.504523 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.506705 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:22.506707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:22.506707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:22.506715 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:22.507949 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:22.508366 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:22.508367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:22.508368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:22.508444 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:22.508701 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:22.508702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:22.508702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:22.508823 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:22.508900 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:22.509184 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 16:32:22.509275 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 16:32:22.509581 21731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.500765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:39891)
0809 16:32:22.500975 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:22.501019 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:22.501604 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:22.501833 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:22.501836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:22.502454 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:22.502648 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.502652 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.503318 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.503318 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.504135 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.504138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.504533 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.504534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.504673 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.505157 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:22.505165 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:22.506592 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:22.508051 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:22.508058 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:22.508060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:22.508284 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:22.508521 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:22.508522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:22.508522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:22.508633 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:22.508900 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:22.508908 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:22.509053 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 16:32:22.509162 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 16:32:22.509323 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":115,"threads_started":1}
I20260809 16:32:22.510102 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 21516
I20260809 16:32:22.510093 21727 heartbeater.cc:347] Connected to a master server at 127.30.23.190:38345
I20260809 16:32:22.510187 21727 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:22.510221 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:32:22.510394 21727 heartbeater.cc:511] Master 127.30.23.190:38345 requested a full tablet report, sending...
I20260809 16:32:22.510409 30814 external_mini_cluster.cc:1428] Setting key 4e05e92ca5718d489aa2805b952169e2
I20260809 16:32:22.510838 21181 ts_manager.cc:194] Registered new tserver with Master: b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163)
I20260809 16:32:22.511253 21181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:39891
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:32:22.536574 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:38345
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:22.620334 21738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:22.620559 21738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:22.620586 21738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:22.621867 21738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:22.621917 21738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:22.621945 21738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:32:22.623543 21738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:38345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:22.623811 21738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:22.624063 21738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:32:22.629733 21738 server_base.cc:1034] running on GCE node
W20260809 16:32:22.629678 21750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:22.629707 21746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:22.629684 21747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:22.630141 21738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:22.630437 21738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:22.631618 21738 hybrid_clock.cc:648] HybridClock initialized: now 1786293142631603 us; error 36 us; skew 500 ppm
Aug 09 16:32:22 dist-test-slave-j2vl krb5kdc[21101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293142, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:22.634593 21738 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:32:22.635139 21738 webserver.cc:460] Webserver started at http://127.30.23.131:33443/ using document root <none> and password file <none>
I20260809 16:32:22.635407 21738 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:22.635483 21738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:22.635634 21738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:32:22.636705 21738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "173bb4832a024ed9ad3cdc56c025e396"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "934da374184e86274f58e8230a4df9f2"
server_key_iv: "ce95de094f85e32bf151ef98c61446ec"
server_key_version: "encryptionkey@0"
I20260809 16:32:22.637087 21738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "173bb4832a024ed9ad3cdc56c025e396"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "934da374184e86274f58e8230a4df9f2"
server_key_iv: "ce95de094f85e32bf151ef98c61446ec"
server_key_version: "encryptionkey@0"
I20260809 16:32:22.638562 21738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:32:22.639470 21759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:22.639706 21738 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:22.639808 21738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "173bb4832a024ed9ad3cdc56c025e396"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "934da374184e86274f58e8230a4df9f2"
server_key_iv: "ce95de094f85e32bf151ef98c61446ec"
server_key_version: "encryptionkey@0"
I20260809 16:32:22.639910 21738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:22.647177 21738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:22.649565 21738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:22.649734 21738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:22.649988 21738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:22.650317 21738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:22.650372 21738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:22.650418 21738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:22.650457 21738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:22.656769 21738 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:39977
I20260809 16:32:22.656808 21933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:39977 every 8 connection(s)
I20260809 16:32:22.657174 21738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:32:22.662098 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 21738
I20260809 16:32:22.662199 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:32:22.662364 30814 external_mini_cluster.cc:1428] Setting key b967895e3264ac0d6572c2092067d3d8
Aug 09 16:32:22 dist-test-slave-j2vl krb5kdc[21101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293142, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:22.666985 21245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.658107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:58719 (local address 127.30.23.190:38345)
0809 16:32:22.658155 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:22.658159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:22.658398 (+ 239us) server_negotiation.cc:407] Connection header received
0809 16:32:22.658968 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:22.658970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:22.658985 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:22.659037 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:22.660015 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.660522 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.661576 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.661770 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.661856 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.664260 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:22.664261 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:22.664262 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:22.664269 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:22.665605 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:22.666007 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:22.666008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:22.666008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:22.666069 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:22.666330 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:22.666331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:22.666331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:22.666449 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:22.666526 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:22.666798 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 16:32:22.666881 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:32:22.667184 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.658087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:58719)
0809 16:32:22.658267 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:22.658316 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:22.658840 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:22.659105 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:22.659109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:22.659720 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:22.659912 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.659916 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.660620 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.660621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.661488 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.661490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.661867 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.661868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.662020 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.662575 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:22.662586 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:22.664162 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:22.665684 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:22.665692 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:22.665695 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:22.665915 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:22.666137 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:22.666138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:22.666139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:22.666253 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:22.666515 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:22.666523 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:22.666666 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 16:32:22.666776 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 16:32:22.666938 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260809 16:32:22.668550 21935 heartbeater.cc:347] Connected to a master server at 127.30.23.190:38345
I20260809 16:32:22.668663 21935 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:22.668851 21935 heartbeater.cc:511] Master 127.30.23.190:38345 requested a full tablet report, sending...
I20260809 16:32:22.669286 21181 ts_manager.cc:194] Registered new tserver with Master: 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977)
I20260809 16:32:22.669721 21181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:58719
I20260809 16:32:22.674145 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:32:22.681975 21245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.675568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60178 (local address 127.30.23.190:38345)
0809 16:32:22.675656 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:22.675659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:22.675753 (+ 94us) server_negotiation.cc:407] Connection header received
0809 16:32:22.675867 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:22.675868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:22.675879 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:22.675934 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:22.676506 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.676994 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.677824 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.678033 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.678119 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.679363 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:22.679364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:22.679365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:22.679372 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:22.680714 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:22.681077 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:22.681079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:22.681079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:22.681138 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:22.681367 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:22.681368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:22.681368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:22.681537 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:22.681654 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:22.681788 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 16:32:22.681870 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 16:32:22.684583 21181 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:32:22.686609 21181 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:32:22.693825 21953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.689327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56012 (local address 127.30.23.129:46363)
0809 16:32:22.689533 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:22.689538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:22.689550 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:32:22.689604 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:22.689606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:22.689640 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:22.689738 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:22.690165 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.691314 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.692726 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.693321 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.693460 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.693536 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:22.693695 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 16:32:22.693741 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":137,"threads_started":1}
I20260809 16:32:22.693811 21952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.689154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:46363 (local address 127.0.0.1:56012)
0809 16:32:22.689281 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:22.689325 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:22.689419 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:22.689985 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:22.689988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:22.689990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:22.690068 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.690070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.691421 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.691422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.692622 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.692624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.693463 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.693464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.693558 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.693572 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:32:22.693659 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:32:22.693720 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260809 16:32:22.694017 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.689761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51988 (local address 127.30.23.131:39977)
0809 16:32:22.690045 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:22.690050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:22.690059 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:32:22.690094 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:22.690096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:22.690112 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:22.690222 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:22.690360 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.691155 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.692738 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.693466 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.693539 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.693593 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:22.693862 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 16:32:22.693903 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":111,"threads_started":1}
I20260809 16:32:22.694240 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.689875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44826 (local address 127.30.23.130:42163)
0809 16:32:22.690352 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:22.690355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:22.690365 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:32:22.690403 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:22.690405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:22.690419 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:22.690515 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:22.690623 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.691593 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.692695 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.693473 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.693539 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.693593 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:22.693862 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 16:32:22.694045 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":96,"threads_started":1}
I20260809 16:32:22.694588 21398 tablet_service.cc:1467] Processing CreateTablet for tablet f20027baa05f4866af34a56f3624af26 (DEFAULT_TABLE table=test-table [id=beaa287e3f56480dab97dc2d7e27094d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:32:22.694749 21955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.689640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:39977 (local address 127.0.0.1:51988)
0809 16:32:22.689862 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:22.689872 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:22.689920 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:22.690227 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:22.690228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:22.690230 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:22.690278 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.690279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.691259 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.691261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.692621 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.692624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.693636 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.693636 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.693705 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.693710 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:32:22.693762 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:32:22.693815 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":43,"thread_start_us":68,"threads_started":1}
I20260809 16:32:22.694816 21954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.689431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:42163 (local address 127.0.0.1:44826)
0809 16:32:22.689567 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:22.689577 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:22.689617 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:22.690524 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:22.690525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:22.690527 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:22.690562 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.690564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.691689 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.691690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.692591 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.692594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.693634 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.693636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.693705 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.693712 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:32:22.693762 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:32:22.693815 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":5,"thread_start_us":101,"threads_started":1}
I20260809 16:32:22.695025 21398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f20027baa05f4866af34a56f3624af26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:22.695595 21836 tablet_service.cc:1467] Processing CreateTablet for tablet f20027baa05f4866af34a56f3624af26 (DEFAULT_TABLE table=test-table [id=beaa287e3f56480dab97dc2d7e27094d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:32:22.695940 21836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f20027baa05f4866af34a56f3624af26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:22.697746 21626 tablet_service.cc:1467] Processing CreateTablet for tablet f20027baa05f4866af34a56f3624af26 (DEFAULT_TABLE table=test-table [id=beaa287e3f56480dab97dc2d7e27094d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:32:22.698087 21626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f20027baa05f4866af34a56f3624af26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:22.698674 21960 tablet_bootstrap.cc:492] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Bootstrap starting.
I20260809 16:32:22.699229 21961 tablet_bootstrap.cc:492] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396: Bootstrap starting.
I20260809 16:32:22.699446 21960 tablet_bootstrap.cc:654] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:22.699915 21961 tablet_bootstrap.cc:654] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:22.700028 21960 log.cc:824] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:22.700503 21961 log.cc:824] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:22.701038 21960 tablet_bootstrap.cc:492] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: No bootstrap required, opened a new log
I20260809 16:32:22.701133 21960 ts_tablet_manager.cc:1397] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:32:22.701545 21965 tablet_bootstrap.cc:492] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042: Bootstrap starting.
I20260809 16:32:22.702139 21965 tablet_bootstrap.cc:654] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:22.702586 21965 log.cc:824] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:22.702884 21960 raft_consensus.cc:348] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } }
I20260809 16:32:22.703032 21960 raft_consensus.cc:374] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:22.703078 21960 raft_consensus.cc:729] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b8ed3a2424f40d9929e7f62be54ecbf, State: Initialized, Role: FOLLOWER
I20260809 16:32:22.703229 21960 consensus_queue.cc:260] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } }
I20260809 16:32:22.703293 21965 tablet_bootstrap.cc:492] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042: No bootstrap required, opened a new log
I20260809 16:32:22.703359 21965 ts_tablet_manager.cc:1397] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:32:22.703536 21960 ts_tablet_manager.cc:1428] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:32:22.703558 21508 heartbeater.cc:503] Master 127.30.23.190:38345 was elected leader, sending a full tablet report...
I20260809 16:32:22.704141 21961 tablet_bootstrap.cc:492] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396: No bootstrap required, opened a new log
I20260809 16:32:22.704217 21961 ts_tablet_manager.cc:1397] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 16:32:22.705034 21965 raft_consensus.cc:348] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } }
I20260809 16:32:22.705201 21965 raft_consensus.cc:374] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:22.705248 21965 raft_consensus.cc:729] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b275b0d7161a4602a97533b6445cf042, State: Initialized, Role: FOLLOWER
I20260809 16:32:22.705390 21965 consensus_queue.cc:260] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } }
I20260809 16:32:22.705678 21965 ts_tablet_manager.cc:1428] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:32:22.705710 21727 heartbeater.cc:503] Master 127.30.23.190:38345 was elected leader, sending a full tablet report...
I20260809 16:32:22.706323 21961 raft_consensus.cc:348] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } }
I20260809 16:32:22.706475 21961 raft_consensus.cc:374] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:22.706512 21961 raft_consensus.cc:729] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173bb4832a024ed9ad3cdc56c025e396, State: Initialized, Role: FOLLOWER
I20260809 16:32:22.706638 21961 consensus_queue.cc:260] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } }
I20260809 16:32:22.706940 21961 ts_tablet_manager.cc:1428] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:32:22.706949 21935 heartbeater.cc:503] Master 127.30.23.190:38345 was elected leader, sending a full tablet report...
W20260809 16:32:22.750602 21730 tablet.cc:2369] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:32:22.819103 21976 raft_consensus.cc:482] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:32:22.819249 21976 raft_consensus.cc:504] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } }
I20260809 16:32:22.819648 21976 leader_election.cc:288] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b8ed3a2424f40d9929e7f62be54ecbf (127.30.23.129:46363), 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977)
I20260809 16:32:22.822628 21731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.819892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:46363 (local address 127.30.23.130:57047)
0809 16:32:22.820011 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:22.820028 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:22.820100 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:22.820237 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:22.820237 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:22.820239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:22.820326 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.820327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.820998 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.820999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.821804 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.821806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.822371 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.822372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.822447 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.822450 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:32:22.822517 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:32:22.822555 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 16:32:22.822628 21953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.820013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:57047 (local address 127.30.23.129:46363)
0809 16:32:22.820105 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:22.820107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:22.820118 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:32:22.820169 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:22.820170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:22.820180 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:22.820229 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:22.820407 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.820908 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.821903 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.822269 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.822368 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.822378 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:22.822522 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 16:32:22.822561 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 16:32:22.822916 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.820139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:53205 (local address 127.30.23.131:39977)
0809 16:32:22.820199 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:22.820201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:22.820329 (+ 128us) server_negotiation.cc:407] Connection header received
0809 16:32:22.820406 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:22.820407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:22.820418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:22.820496 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:22.820656 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.821123 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.822104 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.822496 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.822595 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.822603 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:22.822802 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 16:32:22.822852 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:32:22.823069 21426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f20027baa05f4866af34a56f3624af26" candidate_uuid: "b275b0d7161a4602a97533b6445cf042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" is_pre_election: true
I20260809 16:32:22.823176 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.820045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:39977 (local address 127.30.23.130:53205)
0809 16:32:22.820251 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:22.820260 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:22.820342 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:22.820501 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:22.820502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:22.820505 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:22.820581 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.820582 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.821210 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.821211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.822011 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:22.822013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.822597 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.822598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:22.822674 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.822678 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:32:22.823056 (+ 378us) client_negotiation.cc:240] Negotiation successful
0809 16:32:22.823102 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":147,"threads_started":1}
I20260809 16:32:22.823276 21426 raft_consensus.cc:2454] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b275b0d7161a4602a97533b6445cf042 in term 0.
I20260809 16:32:22.823505 21866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f20027baa05f4866af34a56f3624af26" candidate_uuid: "b275b0d7161a4602a97533b6445cf042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173bb4832a024ed9ad3cdc56c025e396" is_pre_election: true
I20260809 16:32:22.823648 21548 leader_election.cc:302] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b8ed3a2424f40d9929e7f62be54ecbf, b275b0d7161a4602a97533b6445cf042; no voters:
I20260809 16:32:22.823688 21866 raft_consensus.cc:2454] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b275b0d7161a4602a97533b6445cf042 in term 0.
I20260809 16:32:22.823791 21976 raft_consensus.cc:2781] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:32:22.823853 21976 raft_consensus.cc:482] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:32:22.823879 21976 raft_consensus.cc:3037] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:22.824599 21976 raft_consensus.cc:504] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } }
I20260809 16:32:22.824738 21976 leader_election.cc:288] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [CANDIDATE]: Term 1 election: Requested vote from peers 0b8ed3a2424f40d9929e7f62be54ecbf (127.30.23.129:46363), 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977)
I20260809 16:32:22.824889 21426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f20027baa05f4866af34a56f3624af26" candidate_uuid: "b275b0d7161a4602a97533b6445cf042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf"
I20260809 16:32:22.824939 21866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f20027baa05f4866af34a56f3624af26" candidate_uuid: "b275b0d7161a4602a97533b6445cf042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173bb4832a024ed9ad3cdc56c025e396"
I20260809 16:32:22.825001 21866 raft_consensus.cc:3037] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:22.825001 21426 raft_consensus.cc:3037] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:22.825948 21866 raft_consensus.cc:2454] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b275b0d7161a4602a97533b6445cf042 in term 1.
I20260809 16:32:22.825963 21426 raft_consensus.cc:2454] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b275b0d7161a4602a97533b6445cf042 in term 1.
I20260809 16:32:22.826167 21548 leader_election.cc:302] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173bb4832a024ed9ad3cdc56c025e396, b275b0d7161a4602a97533b6445cf042; no voters:
I20260809 16:32:22.826267 21976 raft_consensus.cc:2781] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:32:22.826434 21976 raft_consensus.cc:686] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [term 1 LEADER]: Becoming Leader. State: Replica: b275b0d7161a4602a97533b6445cf042, State: Running, Role: LEADER
I20260809 16:32:22.826576 21976 consensus_queue.cc:237] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } }
W20260809 16:32:22.827389 21509 tablet.cc:2369] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:32:22.827433 21181 catalog_manager.cc:5495] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 reported cstate change: term changed from 0 to 1, leader changed from <none> to b275b0d7161a4602a97533b6445cf042 (127.30.23.130). New cstate: current_term: 1 leader_uuid: "b275b0d7161a4602a97533b6445cf042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } health_report { overall_health: UNKNOWN } } }
I20260809 16:32:22.838119 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.835627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44838 (local address 127.30.23.130:42163)
0809 16:32:22.835689 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:22.835691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:22.835697 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:32:22.835736 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:22.835736 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:22.835754 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:22.835825 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:32:22.835996 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.836465 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.837231 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:22.837482 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:22.837557 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:22.837759 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:32:22.837862 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:32:22.837945 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:22.838023 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 16:32:22.838042 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:32:22.840155 21866 raft_consensus.cc:1261] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396 [term 1 FOLLOWER]: Refusing update from remote peer b275b0d7161a4602a97533b6445cf042: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:32:22.840176 21426 raft_consensus.cc:1261] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Refusing update from remote peer b275b0d7161a4602a97533b6445cf042: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:32:22.840595 21982 consensus_queue.cc:1035] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:32:22.840744 21976 consensus_queue.cc:1035] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:32:22.843328 21986 mvcc.cc:205] Tried to move back new op lower bound from 7316656713069703168 to 7316656713018085376. Current Snapshot: MvccSnapshot[applied={T|T < 7316656713069703168}]
I20260809 16:32:22.843451 21987 mvcc.cc:205] Tried to move back new op lower bound from 7316656713069703168 to 7316656713018085376. Current Snapshot: MvccSnapshot[applied={T|T < 7316656713069703168}]
I20260809 16:32:22.846450 21183 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:32:22.846639 21183 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:32:22.847461 21183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Sending DeleteTablet for 3 replicas of tablet f20027baa05f4866af34a56f3624af26
I20260809 16:32:22.847813 21398 tablet_service.cc:1514] Processing DeleteTablet for tablet f20027baa05f4866af34a56f3624af26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:32:22 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:56012
I20260809 16:32:22.847852 21836 tablet_service.cc:1514] Processing DeleteTablet for tablet f20027baa05f4866af34a56f3624af26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:32:22 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:51988
I20260809 16:32:22.848119 21626 tablet_service.cc:1514] Processing DeleteTablet for tablet f20027baa05f4866af34a56f3624af26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:32:22 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:44826
I20260809 16:32:22.848156 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 21286
I20260809 16:32:22.848356 22006 tablet_replica.cc:330] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042: stopping tablet replica
I20260809 16:32:22.848505 22006 raft_consensus.cc:2229] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:32:22.851027 22006 raft_consensus.cc:2258] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:32:22.854640 22006 ts_tablet_manager.cc:1905] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:32:22.856917 22006 ts_tablet_manager.cc:1918] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:32:22.857079 22006 log.cc:1196] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/f20027baa05f4866af34a56f3624af26
I20260809 16:32:22.857494 22006 ts_tablet_manager.cc:1939] T f20027baa05f4866af34a56f3624af26 P b275b0d7161a4602a97533b6445cf042: Deleting consensus metadata
I20260809 16:32:22.858067 21156 catalog_manager.cc:4841] TS b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163): tablet f20027baa05f4866af34a56f3624af26 (table test-table [id=beaa287e3f56480dab97dc2d7e27094d]) successfully deleted
I20260809 16:32:22.860697 22003 tablet_replica.cc:330] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396: stopping tablet replica
I20260809 16:32:22.860875 22003 raft_consensus.cc:2229] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:32:22.861020 22003 raft_consensus.cc:2258] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:32:22.861543 22003 ts_tablet_manager.cc:1905] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:32:22.863250 22003 ts_tablet_manager.cc:1918] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:32:22.863351 22003 log.cc:1196] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/f20027baa05f4866af34a56f3624af26
I20260809 16:32:22.863618 22003 ts_tablet_manager.cc:1939] T f20027baa05f4866af34a56f3624af26 P 173bb4832a024ed9ad3cdc56c025e396: Deleting consensus metadata
I20260809 16:32:22.864110 21156 catalog_manager.cc:4841] TS 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977): tablet f20027baa05f4866af34a56f3624af26 (table test-table [id=beaa287e3f56480dab97dc2d7e27094d]) successfully deleted
W20260809 16:32:22.868117 21156 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:46363 (error 108)
I20260809 16:32:22.868777 21954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:22.868636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:46363 (local address 127.0.0.1:56022)
0809 16:32:22.868699 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:22.868745 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:46363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 16:32:22.868837 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 21516
W20260809 16:32:22.871768 21156 catalog_manager.cc:4584] TS 0b8ed3a2424f40d9929e7f62be54ecbf (127.30.23.129:46363): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f20027baa05f4866af34a56f3624af26: Network error: Client connection negotiation failed: client connection to 127.30.23.129:46363: connect: Connection refused (error 111)
I20260809 16:32:22.880627 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 21738
I20260809 16:32:22.898876 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 21121
I20260809 16:32:22.915308 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:38345
--webserver_interface=127.30.23.190
--webserver_port=40149
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:38345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:23.008147 22011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:23.008415 22011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:23.008455 22011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:23.009617 22011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:32:23.009686 22011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:23.009711 22011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:23.009732 22011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:32:23.009761 22011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:32:23.011256 22011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:38345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:38345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=40149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:23.011559 22011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:23.011796 22011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:23.021155 22023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:23.021137 22021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:23.021137 22020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:23.021449 22011 server_base.cc:1034] running on GCE node
I20260809 16:32:23.021780 22011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:23.022068 22011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:23.023216 22011 hybrid_clock.cc:648] HybridClock initialized: now 1786293143023201 us; error 26 us; skew 500 ppm
Aug 09 16:32:23 dist-test-slave-j2vl krb5kdc[21101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293143, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:23.025990 22011 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:32:23.026451 22011 webserver.cc:460] Webserver started at http://127.30.23.190:40149/ using document root <none> and password file <none>
I20260809 16:32:23.026718 22011 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:23.026782 22011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:23.028184 22011 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:32:23.028879 22033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:23.029074 22011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:23.029188 22011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "fc859467c0b846f78b48e10905f6db67"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "f45fce56ee31a0cd111a3dc5235a5d8f"
server_key_iv: "217d377993f3640954447347e052da9d"
server_key_version: "encryptionkey@0"
I20260809 16:32:23.029661 22011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:23.044018 22011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:23.046160 22011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:23.046327 22011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:23.050776 22011 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:38345
I20260809 16:32:23.050897 22114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:38345 every 8 connection(s)
I20260809 16:32:23.051266 22011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:32:23.051930 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 22011
I20260809 16:32:23.052239 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:46363
--local_ip_for_outbound_sockets=127.30.23.129
--tserver_master_addrs=127.30.23.190:38345
--webserver_port=37311
--webserver_interface=127.30.23.129
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:32:23.053246 22117 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:32:23.054128 22117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Bootstrap starting.
I20260809 16:32:23.065258 22117 log.cc:824] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:23.068363 22117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:32:23.068783 22117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Bootstrap complete.
I20260809 16:32:23.072372 22117 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } }
I20260809 16:32:23.072703 22117 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc859467c0b846f78b48e10905f6db67, State: Initialized, Role: FOLLOWER
I20260809 16:32:23.072866 22117 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } }
I20260809 16:32:23.072934 22117 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:32:23.072975 22117 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:32:23.073024 22117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:32:23.074633 22117 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } }
I20260809 16:32:23.074766 22117 leader_election.cc:302] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc859467c0b846f78b48e10905f6db67; no voters:
I20260809 16:32:23.074977 22117 leader_election.cc:288] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:32:23.075101 22126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:32:23.075243 22117 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:32:23.075280 22126 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 2 LEADER]: Becoming Leader. State: Replica: fc859467c0b846f78b48e10905f6db67, State: Running, Role: LEADER
I20260809 16:32:23.075384 22126 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } }
I20260809 16:32:23.075664 22126 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc859467c0b846f78b48e10905f6db67. Latest consensus state: current_term: 2 leader_uuid: "fc859467c0b846f78b48e10905f6db67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } } }
I20260809 16:32:23.075805 22126 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:23.075665 22127 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fc859467c0b846f78b48e10905f6db67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } } }
I20260809 16:32:23.075971 22127 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:23.076076 22140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:32:23.076928 22140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:32:23.077069 22140 catalog_manager.cc:1259] Loaded cluster ID: 6545bd167ca74307ae9cbeea61b5e4ae
I20260809 16:32:23.077113 22140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:32:23.077543 22140 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 16:32:23.080327 22140 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 16:32:23.086831 22140 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:32:23.087067 22140 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Loaded TSK: 0
I20260809 16:32:23.087289 22140 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:32:23.154690 22121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:23.154959 22121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:23.154997 22121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:23.156149 22121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:23.156215 22121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:23.156261 22121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:32:23.157754 22121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:46363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=37311
--tserver_master_addrs=127.30.23.190:38345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:23.158027 22121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:23.158263 22121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:23.164582 22170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:23.164605 22174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:23.164714 22169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:23.165032 22121 server_base.cc:1034] running on GCE node
I20260809 16:32:23.165243 22121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:23.165522 22121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:23.166692 22121 hybrid_clock.cc:648] HybridClock initialized: now 1786293143166679 us; error 32 us; skew 500 ppm
Aug 09 16:32:23 dist-test-slave-j2vl krb5kdc[21101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293143, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:23.169415 22121 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:32:23.169925 22121 webserver.cc:460] Webserver started at http://127.30.23.129:37311/ using document root <none> and password file <none>
I20260809 16:32:23.170169 22121 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:23.170255 22121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:23.171663 22121 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:32:23.172406 22188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:23.172593 22121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:23.172709 22121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "0b8ed3a2424f40d9929e7f62be54ecbf"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "43bf4d71816d5c6186e23449ecd7e3de"
server_key_iv: "e8da4a38acc49f2aa391d8da782cc21f"
server_key_version: "encryptionkey@0"
I20260809 16:32:23.173080 22121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:23.193221 22121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:23.195576 22121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:23.195737 22121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:23.196028 22121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:23.196537 22197 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:32:23.198487 22121 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:32:23.198558 22121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:32:23.198603 22121 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:32:23.199294 22121 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:32:23.199352 22121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:32:23.199409 22197 tablet_bootstrap.cc:492] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Bootstrap starting.
I20260809 16:32:23.206854 22121 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:46363
I20260809 16:32:23.207242 22121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:32:23.209544 22376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:46363 every 8 connection(s)
I20260809 16:32:23.210261 22197 log.cc:824] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:23.212028 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 22121
I20260809 16:32:23.212342 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:42163
--local_ip_for_outbound_sockets=127.30.23.130
--tserver_master_addrs=127.30.23.190:38345
--webserver_port=42043
--webserver_interface=127.30.23.130
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:32:23.212858 22197 tablet_bootstrap.cc:492] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:32:23.213204 22197 tablet_bootstrap.cc:492] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Bootstrap complete.
I20260809 16:32:23.214497 22197 ts_tablet_manager.cc:1397] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260809 16:32:23.215957 22197 raft_consensus.cc:348] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } }
I20260809 16:32:23.216334 22197 raft_consensus.cc:729] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b8ed3a2424f40d9929e7f62be54ecbf, State: Initialized, Role: FOLLOWER
I20260809 16:32:23.216521 22197 consensus_queue.cc:260] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } }
I20260809 16:32:23.216842 22197 ts_tablet_manager.cc:1428] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
Aug 09 16:32:23 dist-test-slave-j2vl krb5kdc[21101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293143, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:23.223312 22385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.208089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:56957 (local address 127.30.23.190:38345)
0809 16:32:23.211503 (+ 3414us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:23.211512 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:23.211551 (+ 39us) server_negotiation.cc:407] Connection header received
0809 16:32:23.211677 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:23.211683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:23.211734 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:23.211901 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:23.212912 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.214363 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.215968 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.216789 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.216888 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.219767 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:23.219771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:23.219771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:23.219783 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:23.221530 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:23.221970 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:23.221973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:23.221974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:23.222070 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:23.222334 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:23.222335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:23.222335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:23.222569 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:23.222665 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:23.223003 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 16:32:23.223129 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3359,"thread_start_us":89,"threads_started":1}
I20260809 16:32:23.223312 22382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.208002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:56957)
0809 16:32:23.210568 (+ 2566us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:23.210616 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:23.211233 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:23.212110 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:23.212116 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:23.212504 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:23.212777 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.212782 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.214531 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.214532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.215832 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.215835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.217456 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.217457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.217614 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.218229 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:23.218238 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:23.219649 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:23.221635 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:23.221643 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:23.221645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:23.221871 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:23.222151 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:23.222152 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:23.222152 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:23.222264 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:23.222656 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:23.222663 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:23.222801 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 16:32:23.222923 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 16:32:23.223113 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2522,"thread_start_us":98,"threads_started":1}
I20260809 16:32:23.224480 22379 heartbeater.cc:347] Connected to a master server at 127.30.23.190:38345
I20260809 16:32:23.224602 22379 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:23.224782 22379 heartbeater.cc:511] Master 127.30.23.190:38345 requested a full tablet report, sending...
I20260809 16:32:23.225572 22065 ts_manager.cc:194] Registered new tserver with Master: 0b8ed3a2424f40d9929e7f62be54ecbf (127.30.23.129:46363)
I20260809 16:32:23.226763 22065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:56957
I20260809 16:32:23.227623 22379 heartbeater.cc:503] Master 127.30.23.190:38345 was elected leader, sending a full tablet report...
I20260809 16:32:23.231719 22395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.226171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:46363 (local address 127.0.0.1:56026)
0809 16:32:23.228101 (+ 1930us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:23.228139 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:23.228215 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:23.228508 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:23.228511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:23.228513 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:23.228565 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.228566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.229831 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.229831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.230605 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.230608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.231495 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.231496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.231557 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.231567 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 16:32:23.231627 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:32:23.231666 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1900,"thread_start_us":98,"threads_started":1}
I20260809 16:32:23.231772 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.226409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56026 (local address 127.30.23.129:46363)
0809 16:32:23.228354 (+ 1945us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:23.228359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:23.228368 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:32:23.228400 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:23.228402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:23.228417 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:23.228490 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:23.228644 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.229755 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.230691 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.231399 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.231473 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.231524 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:23.231660 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 16:32:23.231700 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1913,"thread_start_us":81,"threads_started":1}
I20260809 16:32:23.232170 22254 tablet_service.cc:1514] Processing DeleteTablet for tablet f20027baa05f4866af34a56f3624af26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:32:22 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:56026
I20260809 16:32:23.232756 22397 tablet_replica.cc:330] stopping tablet replica
I20260809 16:32:23.232877 22397 raft_consensus.cc:2229] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:32:23.232975 22397 raft_consensus.cc:2258] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:32:23.233289 22397 ts_tablet_manager.cc:1905] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:32:23.235322 22397 ts_tablet_manager.cc:1918] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:32:23.235417 22397 log.cc:1196] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal/wals/f20027baa05f4866af34a56f3624af26
I20260809 16:32:23.235666 22397 ts_tablet_manager.cc:1939] T f20027baa05f4866af34a56f3624af26 P 0b8ed3a2424f40d9929e7f62be54ecbf: Deleting consensus metadata
I20260809 16:32:23.236186 22038 catalog_manager.cc:4841] TS 0b8ed3a2424f40d9929e7f62be54ecbf (127.30.23.129:46363): tablet f20027baa05f4866af34a56f3624af26 (table test-table [id=beaa287e3f56480dab97dc2d7e27094d]) successfully deleted
W20260809 16:32:23.321882 22390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:23.322110 22390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:23.322151 22390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:23.323311 22390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:23.323377 22390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:23.323438 22390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:32:23.325031 22390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:42163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=42043
--tserver_master_addrs=127.30.23.190:38345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:23.325299 22390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:23.325557 22390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:23.330739 22407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:23.330777 22414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:23.331053 22412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:23.331005 22390 server_base.cc:1034] running on GCE node
I20260809 16:32:23.331274 22390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:23.331559 22390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:23.332728 22390 hybrid_clock.cc:648] HybridClock initialized: now 1786293143332706 us; error 32 us; skew 500 ppm
Aug 09 16:32:23 dist-test-slave-j2vl krb5kdc[21101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293143, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:23.336269 22390 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:32:23.336901 22390 webserver.cc:460] Webserver started at http://127.30.23.130:42043/ using document root <none> and password file <none>
I20260809 16:32:23.337209 22390 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:23.337296 22390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:23.339303 22390 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:32:23.340122 22424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:23.340392 22390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:23.340498 22390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "b275b0d7161a4602a97533b6445cf042"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "642fc3068f5ba762b088aa71bf0b43c8"
server_key_iv: "474022df76c511b08cfd5a6fe422abad"
server_key_version: "encryptionkey@0"
I20260809 16:32:23.341033 22390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:23.359939 22390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:23.363044 22390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:23.363225 22390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:23.363529 22390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:23.363931 22390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:23.363983 22390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:23.364022 22390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:23.364045 22390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:23.371063 22390 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:42163
I20260809 16:32:23.371147 22605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:42163 every 8 connection(s)
I20260809 16:32:23.371500 22390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:32:23 dist-test-slave-j2vl krb5kdc[21101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293143, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:23.380293 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 22390
I20260809 16:32:23.380575 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:39977
--local_ip_for_outbound_sockets=127.30.23.131
--tserver_master_addrs=127.30.23.190:38345
--webserver_port=33443
--webserver_interface=127.30.23.131
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:32:23.382421 22385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.372237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:53459 (local address 127.30.23.190:38345)
0809 16:32:23.372300 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:23.372302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:23.372504 (+ 202us) server_negotiation.cc:407] Connection header received
0809 16:32:23.373048 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:23.373050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:23.373063 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:23.373132 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:23.374361 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.375007 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.376170 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.376440 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.376506 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.379042 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:23.379044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:23.379044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:23.379054 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:23.380860 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:23.381326 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:23.381328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:23.381328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:23.381424 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:23.381705 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:23.381706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:23.381706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:23.381838 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:23.381914 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:23.382206 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 16:32:23.382309 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:32:23.382799 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.372189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:53459)
0809 16:32:23.372361 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:23.372406 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:23.372922 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:23.373333 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:23.373337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:23.374026 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:23.374223 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.374228 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.375136 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.375138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.376087 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.376089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.376524 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.376525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.376839 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.377470 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:23.377479 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:23.378935 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:23.380973 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:23.380982 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:23.380986 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:23.381240 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:23.381504 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:23.381505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:23.381506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:23.381631 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:23.381916 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:23.381924 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:23.382072 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 16:32:23.382184 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 16:32:23.382360 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260809 16:32:23.383701 22606 heartbeater.cc:347] Connected to a master server at 127.30.23.190:38345
I20260809 16:32:23.383821 22606 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:23.384035 22606 heartbeater.cc:511] Master 127.30.23.190:38345 requested a full tablet report, sending...
I20260809 16:32:23.384502 22065 ts_manager.cc:194] Registered new tserver with Master: b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163)
I20260809 16:32:23.384879 22065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:53459
W20260809 16:32:23.467749 22611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:23.468004 22611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:23.468050 22611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:23.469286 22611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:23.469377 22611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:23.469430 22611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:32:23.470971 22611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:39977
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=33443
--tserver_master_addrs=127.30.23.190:38345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:23.471262 22611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:23.471519 22611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:23.477310 22620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:23.477304 22619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:23.477550 22611 server_base.cc:1034] running on GCE node
W20260809 16:32:23.477663 22622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:23.477911 22611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:23.478188 22611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:23.479378 22611 hybrid_clock.cc:648] HybridClock initialized: now 1786293143479359 us; error 32 us; skew 500 ppm
Aug 09 16:32:23 dist-test-slave-j2vl krb5kdc[21101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293143, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:23.482297 22611 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:32:23.482779 22611 webserver.cc:460] Webserver started at http://127.30.23.131:33443/ using document root <none> and password file <none>
I20260809 16:32:23.483049 22611 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:23.483140 22611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:23.484671 22611 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:32:23.485491 22630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:23.485821 22611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:32:23.486049 22611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "173bb4832a024ed9ad3cdc56c025e396"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "934da374184e86274f58e8230a4df9f2"
server_key_iv: "ce95de094f85e32bf151ef98c61446ec"
server_key_version: "encryptionkey@0"
I20260809 16:32:23.486459 22611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:23.499974 22611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:23.502877 22611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:23.503182 22611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:23.503482 22611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:23.503830 22611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:23.503890 22611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:23.503933 22611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:23.503964 22611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:23.510640 22611 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:39977
I20260809 16:32:23.510677 22821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:39977 every 8 connection(s)
I20260809 16:32:23.511107 22611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:32:23.517702 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 22611
Aug 09 16:32:23 dist-test-slave-j2vl krb5kdc[21101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293143, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:23.523258 22385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.512207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:45919 (local address 127.30.23.190:38345)
0809 16:32:23.512307 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:23.512309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:23.512734 (+ 425us) server_negotiation.cc:407] Connection header received
0809 16:32:23.513233 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:23.513234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:23.513246 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:23.513309 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:23.514718 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.515245 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.516523 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.516724 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.516794 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.519700 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:23.519701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:23.519702 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:23.519710 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:23.521538 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:23.522100 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:23.522101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:23.522101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:23.522157 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:23.522464 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:23.522465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:23.522465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:23.522606 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:23.522686 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:23.523057 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 16:32:23.523155 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 16:32:23.523443 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.512181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:45919)
0809 16:32:23.512379 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:23.512424 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:23.513100 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:23.513443 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:23.513447 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:23.514240 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:23.514579 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.514586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.515358 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.515359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.516406 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.516410 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.516807 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.516808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.516935 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.517690 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:23.517700 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:23.519578 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:23.521613 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:23.521621 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:23.521623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:23.521996 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:23.522237 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:23.522238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:23.522238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:23.522396 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:23.522712 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:23.522726 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:23.522866 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 16:32:23.522976 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 16:32:23.523182 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":120,"threads_started":1}
I20260809 16:32:23.524246 22822 heartbeater.cc:347] Connected to a master server at 127.30.23.190:38345
I20260809 16:32:23.524353 22822 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:23.524585 22822 heartbeater.cc:511] Master 127.30.23.190:38345 requested a full tablet report, sending...
I20260809 16:32:23.524998 22065 ts_manager.cc:194] Registered new tserver with Master: 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977)
I20260809 16:32:23.525528 22065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:45919
I20260809 16:32:23.527263 22825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.518331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60186 (local address 127.30.23.190:38345)
0809 16:32:23.518601 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:23.518603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:23.518671 (+ 68us) server_negotiation.cc:407] Connection header received
0809 16:32:23.518725 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:23.518726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:23.518737 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:23.518798 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:23.519553 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.520201 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.521456 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.521739 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.521817 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.523638 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:23.523639 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:23.523639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:23.523645 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:23.525745 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:23.526550 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:23.526551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:23.526551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:23.526610 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:23.526807 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:23.526808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:23.526808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:23.526909 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:23.527029 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:23.527133 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 16:32:23.527198 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":156,"threads_started":1}
I20260809 16:32:23.528016 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:32:23.530998 22825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.528402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60188 (local address 127.30.23.190:38345)
0809 16:32:23.528441 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:23.528452 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:23.528546 (+ 94us) server_negotiation.cc:407] Connection header received
0809 16:32:23.528609 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:23.528610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:23.528638 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:23.528702 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:32:23.528813 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.529268 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.530240 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.530463 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.530526 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.530716 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:32:23.530820 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:32:23.530885 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:23.530920 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 16:32:23.530943 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:32:23.531291 22065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:32:23.532953 22065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:32:23.535601 22254 tablet_service.cc:1467] Processing CreateTablet for tablet 353ee0fdab9f4f888d6856e58f6d01c5 (DEFAULT_TABLE table=test-table [id=aa8c753718264920bc0ee8e677b4de28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:32:23.535833 22254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353ee0fdab9f4f888d6856e58f6d01c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:23.537467 22197 tablet_bootstrap.cc:492] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf: Bootstrap starting.
I20260809 16:32:23.537832 22197 tablet_bootstrap.cc:654] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:23.539299 22395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.535029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:39977 (local address 127.0.0.1:52004)
0809 16:32:23.535106 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:23.535121 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:23.535170 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:23.535508 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:23.535509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:23.535510 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:23.535561 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.535563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.536945 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.536946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.538011 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.538014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.538977 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.538978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.539084 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.539088 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:32:23.539161 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 16:32:23.539216 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 16:32:23.539351 22833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.535143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52004 (local address 127.30.23.131:39977)
0809 16:32:23.535349 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:23.535355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:23.535366 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:32:23.535407 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:23.535409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:23.535426 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:23.535512 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:23.535623 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.536821 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.538114 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.538867 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.538951 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.539019 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:23.539192 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 16:32:23.539253 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":131,"threads_started":1}
I20260809 16:32:23.539361 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.535317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44840 (local address 127.30.23.130:42163)
0809 16:32:23.535512 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:23.535517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:23.535526 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:32:23.535564 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:23.535575 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:23.535595 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:23.535682 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:23.535800 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.536663 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.538292 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.538971 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.539040 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.539090 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:23.539223 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 16:32:23.539272 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":85,"threads_started":1}
I20260809 16:32:23.539548 22829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.535159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:42163 (local address 127.0.0.1:44840)
0809 16:32:23.535343 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:23.535354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:23.535402 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:23.535679 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:23.535680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:23.535682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:23.535718 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.535749 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.536774 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.536775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.538205 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.538208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.539056 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.539057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.539110 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.539114 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:32:23.539436 (+ 322us) client_negotiation.cc:240] Negotiation successful
0809 16:32:23.539485 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":69,"threads_started":1}
I20260809 16:32:23.539947 22197 tablet_bootstrap.cc:492] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf: No bootstrap required, opened a new log
I20260809 16:32:23.540030 22197 ts_tablet_manager.cc:1397] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260809 16:32:23.540205 22197 raft_consensus.cc:348] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:23.540132 22712 tablet_service.cc:1467] Processing CreateTablet for tablet 353ee0fdab9f4f888d6856e58f6d01c5 (DEFAULT_TABLE table=test-table [id=aa8c753718264920bc0ee8e677b4de28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:32:23.540292 22197 raft_consensus.cc:374] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:23.540343 22197 raft_consensus.cc:729] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b8ed3a2424f40d9929e7f62be54ecbf, State: Initialized, Role: FOLLOWER
I20260809 16:32:23.540270 22501 tablet_service.cc:1467] Processing CreateTablet for tablet 353ee0fdab9f4f888d6856e58f6d01c5 (DEFAULT_TABLE table=test-table [id=aa8c753718264920bc0ee8e677b4de28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:32:23.540400 22197 consensus_queue.cc:260] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:23.540494 22197 ts_tablet_manager.cc:1428] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:23.540534 22712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353ee0fdab9f4f888d6856e58f6d01c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:23.540580 22501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353ee0fdab9f4f888d6856e58f6d01c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:23.544648 22840 tablet_bootstrap.cc:492] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396: Bootstrap starting.
I20260809 16:32:23.545491 22838 tablet_bootstrap.cc:492] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042: Bootstrap starting.
I20260809 16:32:23.546075 22838 tablet_bootstrap.cc:654] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:23.546646 22838 log.cc:824] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:23.546954 22840 tablet_bootstrap.cc:654] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:23.547492 22840 log.cc:824] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:23.547530 22838 tablet_bootstrap.cc:492] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042: No bootstrap required, opened a new log
I20260809 16:32:23.547609 22838 ts_tablet_manager.cc:1397] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:32:23.548339 22840 tablet_bootstrap.cc:492] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396: No bootstrap required, opened a new log
I20260809 16:32:23.548430 22840 ts_tablet_manager.cc:1397] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 16:32:23.549765 22838 raft_consensus.cc:348] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:23.549986 22838 raft_consensus.cc:374] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:23.550025 22838 raft_consensus.cc:729] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b275b0d7161a4602a97533b6445cf042, State: Initialized, Role: FOLLOWER
I20260809 16:32:23.550155 22838 consensus_queue.cc:260] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:23.550494 22838 ts_tablet_manager.cc:1428] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:32:23.550591 22606 heartbeater.cc:503] Master 127.30.23.190:38345 was elected leader, sending a full tablet report...
I20260809 16:32:23.550595 22840 raft_consensus.cc:348] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:23.550720 22840 raft_consensus.cc:374] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:23.550741 22840 raft_consensus.cc:729] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173bb4832a024ed9ad3cdc56c025e396, State: Initialized, Role: FOLLOWER
I20260809 16:32:23.550822 22840 consensus_queue.cc:260] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:23.551133 22840 ts_tablet_manager.cc:1428] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:32:23.551170 22822 heartbeater.cc:503] Master 127.30.23.190:38345 was elected leader, sending a full tablet report...
I20260809 16:32:23.577577 22849 raft_consensus.cc:482] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:32:23.577713 22849 raft_consensus.cc:504] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:23.578137 22849 leader_election.cc:288] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977), 0b8ed3a2424f40d9929e7f62be54ecbf (127.30.23.129:46363)
I20260809 16:32:23.581274 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.578365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:39977 (local address 127.30.23.130:50847)
0809 16:32:23.578446 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:23.578460 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:23.578513 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:23.578690 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:23.578691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:23.578693 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:23.578766 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.578768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.579552 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.579553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.580374 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.580377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.580998 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.581000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.581088 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.581093 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:32:23.581155 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:32:23.581206 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 16:32:23.581298 22833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.578466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:50847 (local address 127.30.23.131:39977)
0809 16:32:23.578521 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:23.578524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:23.578538 (+ 14us) server_negotiation.cc:407] Connection header received
0809 16:32:23.578574 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:23.578575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:23.578586 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:23.578668 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:23.578847 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.579461 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.580470 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.580910 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.580968 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.580976 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:23.581175 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 16:32:23.581216 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:32:23.581511 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.578625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:59241 (local address 127.30.23.129:46363)
0809 16:32:23.578689 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:23.578691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:23.578884 (+ 193us) server_negotiation.cc:407] Connection header received
0809 16:32:23.578929 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:23.578930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:23.578938 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:23.578983 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:23.579139 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.579690 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.580637 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.581087 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.581165 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.581173 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:23.581330 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 16:32:23.581425 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:32:23.581697 22751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353ee0fdab9f4f888d6856e58f6d01c5" candidate_uuid: "b275b0d7161a4602a97533b6445cf042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173bb4832a024ed9ad3cdc56c025e396" is_pre_election: true
I20260809 16:32:23.581840 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.578563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:46363 (local address 127.30.23.130:59241)
0809 16:32:23.578765 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:23.578774 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:23.578877 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:23.579000 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:23.579001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:23.579003 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:23.579072 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.579073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.579770 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.579771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.580547 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:23.580550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.581180 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.581181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:23.581248 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.581252 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:32:23.581309 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:32:23.581748 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":124,"threads_started":1}
I20260809 16:32:23.581871 22751 raft_consensus.cc:2454] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b275b0d7161a4602a97533b6445cf042 in term 0.
I20260809 16:32:23.582187 22427 leader_election.cc:302] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173bb4832a024ed9ad3cdc56c025e396, b275b0d7161a4602a97533b6445cf042; no voters:
I20260809 16:32:23.582156 22291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353ee0fdab9f4f888d6856e58f6d01c5" candidate_uuid: "b275b0d7161a4602a97533b6445cf042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" is_pre_election: true
I20260809 16:32:23.582319 22849 raft_consensus.cc:2781] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:32:23.582370 22849 raft_consensus.cc:482] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:32:23.582433 22849 raft_consensus.cc:3037] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:23.582428 22291 raft_consensus.cc:2454] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b275b0d7161a4602a97533b6445cf042 in term 0.
I20260809 16:32:23.583562 22849 raft_consensus.cc:504] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:23.583719 22849 leader_election.cc:288] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [CANDIDATE]: Term 1 election: Requested vote from peers 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977), 0b8ed3a2424f40d9929e7f62be54ecbf (127.30.23.129:46363)
I20260809 16:32:23.583928 22751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353ee0fdab9f4f888d6856e58f6d01c5" candidate_uuid: "b275b0d7161a4602a97533b6445cf042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173bb4832a024ed9ad3cdc56c025e396"
I20260809 16:32:23.583928 22291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353ee0fdab9f4f888d6856e58f6d01c5" candidate_uuid: "b275b0d7161a4602a97533b6445cf042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf"
I20260809 16:32:23.584040 22291 raft_consensus.cc:3037] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:23.584040 22751 raft_consensus.cc:3037] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:23.585079 22751 raft_consensus.cc:2454] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b275b0d7161a4602a97533b6445cf042 in term 1.
I20260809 16:32:23.585083 22291 raft_consensus.cc:2454] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b275b0d7161a4602a97533b6445cf042 in term 1.
I20260809 16:32:23.585304 22427 leader_election.cc:302] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b8ed3a2424f40d9929e7f62be54ecbf, b275b0d7161a4602a97533b6445cf042; no voters:
I20260809 16:32:23.585489 22849 raft_consensus.cc:2781] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:32:23.585691 22849 raft_consensus.cc:686] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [term 1 LEADER]: Becoming Leader. State: Replica: b275b0d7161a4602a97533b6445cf042, State: Running, Role: LEADER
I20260809 16:32:23.585810 22849 consensus_queue.cc:237] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:23.587119 22064 catalog_manager.cc:5495] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 reported cstate change: term changed from 0 to 1, leader changed from <none> to b275b0d7161a4602a97533b6445cf042 (127.30.23.130). New cstate: current_term: 1 leader_uuid: "b275b0d7161a4602a97533b6445cf042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } health_report { overall_health: UNKNOWN } } }
I20260809 16:32:23.600772 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.598137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44852 (local address 127.30.23.130:42163)
0809 16:32:23.598204 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:23.598206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:23.598215 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:32:23.598266 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:23.598267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:23.598287 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:23.598362 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:32:23.598516 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.599016 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.599791 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:23.600002 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:23.600084 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:23.600323 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:32:23.600477 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:32:23.600596 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:23.600623 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 16:32:23.600647 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:32:23.603221 22291 raft_consensus.cc:1261] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Refusing update from remote peer b275b0d7161a4602a97533b6445cf042: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:32:23.603221 22751 raft_consensus.cc:1261] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396 [term 1 FOLLOWER]: Refusing update from remote peer b275b0d7161a4602a97533b6445cf042: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:32:23.603654 22854 consensus_queue.cc:1035] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:32:23.603811 22849 consensus_queue.cc:1035] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:32:23.606392 22859 mvcc.cc:205] Tried to move back new op lower bound from 7316656716195065856 to 7316656716129079296. Current Snapshot: MvccSnapshot[applied={T|T < 7316656716195065856}]
I20260809 16:32:23.607306 22860 mvcc.cc:205] Tried to move back new op lower bound from 7316656716195065856 to 7316656716129079296. Current Snapshot: MvccSnapshot[applied={T|T < 7316656716195065856}]
I20260809 16:32:23.608606 22065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:32:23.608717 22065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:32:23.609651 22065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Sending DeleteTablet for 3 replicas of tablet 353ee0fdab9f4f888d6856e58f6d01c5
I20260809 16:32:23.610044 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 22121
I20260809 16:32:23.610107 22254 tablet_service.cc:1514] Processing DeleteTablet for tablet 353ee0fdab9f4f888d6856e58f6d01c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:32:23 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:56026
I20260809 16:32:23.610090 22712 tablet_service.cc:1514] Processing DeleteTablet for tablet 353ee0fdab9f4f888d6856e58f6d01c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:32:23 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:52004
I20260809 16:32:23.610132 22501 tablet_service.cc:1514] Processing DeleteTablet for tablet 353ee0fdab9f4f888d6856e58f6d01c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:32:23 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:44840
I20260809 16:32:23.617306 22881 tablet_replica.cc:330] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396: stopping tablet replica
I20260809 16:32:23.617486 22881 raft_consensus.cc:2229] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:32:23.617607 22881 raft_consensus.cc:2258] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:32:23.617774 22882 tablet_replica.cc:330] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042: stopping tablet replica
I20260809 16:32:23.617995 22882 raft_consensus.cc:2229] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:32:23.618093 22882 raft_consensus.cc:2258] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:32:23.618599 22882 ts_tablet_manager.cc:1905] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:32:23.619841 22881 ts_tablet_manager.cc:1905] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:32:23.621388 22881 ts_tablet_manager.cc:1918] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:32:23.621474 22881 log.cc:1196] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/353ee0fdab9f4f888d6856e58f6d01c5
I20260809 16:32:23.621788 22881 ts_tablet_manager.cc:1939] T 353ee0fdab9f4f888d6856e58f6d01c5 P 173bb4832a024ed9ad3cdc56c025e396: Deleting consensus metadata
I20260809 16:32:23.621982 22882 ts_tablet_manager.cc:1918] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:32:23.622062 22882 log.cc:1196] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/353ee0fdab9f4f888d6856e58f6d01c5
I20260809 16:32:23.622396 22038 catalog_manager.cc:4841] TS 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977): tablet 353ee0fdab9f4f888d6856e58f6d01c5 (table test-table [id=aa8c753718264920bc0ee8e677b4de28]) successfully deleted
I20260809 16:32:23.622527 22882 ts_tablet_manager.cc:1939] T 353ee0fdab9f4f888d6856e58f6d01c5 P b275b0d7161a4602a97533b6445cf042: Deleting consensus metadata
I20260809 16:32:23.622937 22038 catalog_manager.cc:4841] TS b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163): tablet 353ee0fdab9f4f888d6856e58f6d01c5 (table test-table [id=aa8c753718264920bc0ee8e677b4de28]) successfully deleted
W20260809 16:32:23.631286 22038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:46363 (error 108)
I20260809 16:32:23.631615 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 22390
I20260809 16:32:23.631898 22829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:23.631761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:46363 (local address 127.0.0.1:56030)
0809 16:32:23.631810 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:23.631853 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:46363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:32:23.632138 22038 catalog_manager.cc:4584] TS 0b8ed3a2424f40d9929e7f62be54ecbf (127.30.23.129:46363): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 353ee0fdab9f4f888d6856e58f6d01c5: Network error: Client connection negotiation failed: client connection to 127.30.23.129:46363: connect: Connection refused (error 111)
I20260809 16:32:23.646085 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 22611
I20260809 16:32:23.659734 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 22011
I20260809 16:32:23.679206 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:38345
--webserver_interface=127.30.23.190
--webserver_port=40149
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:38345
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:23.771930 22890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:23.772207 22890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:23.772244 22890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:23.773532 22890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:32:23.773602 22890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:23.773628 22890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:23.773660 22890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:32:23.773687 22890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:32:23.775480 22890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:38345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:38345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=40149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:23.775780 22890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:23.776028 22890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:23.781684 22899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:23.781863 22900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:23.781697 22903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:23.782131 22890 server_base.cc:1034] running on GCE node
I20260809 16:32:23.782491 22890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:23.782850 22890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:23.784049 22890 hybrid_clock.cc:648] HybridClock initialized: now 1786293143784028 us; error 49 us; skew 500 ppm
Aug 09 16:32:23 dist-test-slave-j2vl krb5kdc[21101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293143, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:23.787410 22890 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:32:23.787981 22890 webserver.cc:460] Webserver started at http://127.30.23.190:40149/ using document root <none> and password file <none>
I20260809 16:32:23.788224 22890 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:23.788302 22890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:23.789830 22890 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:32:23.790624 22914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:23.790830 22890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:23.790974 22890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "fc859467c0b846f78b48e10905f6db67"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "f45fce56ee31a0cd111a3dc5235a5d8f"
server_key_iv: "217d377993f3640954447347e052da9d"
server_key_version: "encryptionkey@0"
I20260809 16:32:23.791349 22890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:23.805739 22890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:23.808183 22890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:23.808378 22890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:23.813047 22890 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:38345
I20260809 16:32:23.813184 22993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:38345 every 8 connection(s)
I20260809 16:32:23.813588 22890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:32:23.815764 22994 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:32:23.815829 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 22890
I20260809 16:32:23.816174 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:46363
--local_ip_for_outbound_sockets=127.30.23.129
--tserver_master_addrs=127.30.23.190:38345
--webserver_port=37311
--webserver_interface=127.30.23.129
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:32:23.817030 22994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Bootstrap starting.
I20260809 16:32:23.826954 22994 log.cc:824] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:23.829919 22994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:32:23.830252 22994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67: Bootstrap complete.
I20260809 16:32:23.832682 22994 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } }
I20260809 16:32:23.832918 22994 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc859467c0b846f78b48e10905f6db67, State: Initialized, Role: FOLLOWER
I20260809 16:32:23.833093 22994 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } }
I20260809 16:32:23.833168 22994 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:32:23.833216 22994 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:32:23.833267 22994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 2 FOLLOWER]: Advancing to term 3
I20260809 16:32:23.834383 22994 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } }
I20260809 16:32:23.834499 22994 leader_election.cc:302] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc859467c0b846f78b48e10905f6db67; no voters:
I20260809 16:32:23.834746 22994 leader_election.cc:288] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 16:32:23.834790 23002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 16:32:23.834910 23002 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [term 3 LEADER]: Becoming Leader. State: Replica: fc859467c0b846f78b48e10905f6db67, State: Running, Role: LEADER
I20260809 16:32:23.835043 22994 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:32:23.835023 23002 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } }
I20260809 16:32:23.835417 23005 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc859467c0b846f78b48e10905f6db67. Latest consensus state: current_term: 3 leader_uuid: "fc859467c0b846f78b48e10905f6db67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } } }
I20260809 16:32:23.835587 23005 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:23.835408 23003 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "fc859467c0b846f78b48e10905f6db67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc859467c0b846f78b48e10905f6db67" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 38345 } } }
I20260809 16:32:23.835879 23003 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc859467c0b846f78b48e10905f6db67 [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:23.836018 23021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:32:23.837149 23021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:32:23.837281 23021 catalog_manager.cc:1259] Loaded cluster ID: 6545bd167ca74307ae9cbeea61b5e4ae
I20260809 16:32:23.837324 23021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 16:32:23.837683 23021 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7dd1c6f9ad google::LogMessage::Fail() at ??:0
@ 0x7f7dd1c7288a google::LogMessage::SendToLog() at ??:0
@ 0x7f7dd1c6f4de google::LogMessage::Flush() at ??:0
@ 0x7f7dd1c70a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7dd9b900f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f7dd9b998b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7dd3b3d0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7dd3b32954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7dd49966db start_thread at ??:0
@ 0x7f7dd076d61f clone at ??:0
W20260809 16:32:23.920325 22999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:23.920732 22999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:23.920807 22999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:23.922614 22999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:23.922693 22999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:23.922829 22999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:32:23.924629 22999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:46363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=37311
--tserver_master_addrs=127.30.23.190:38345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:23.924942 22999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:23.925200 22999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:23.932178 23056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:23.932302 23055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:23.932464 23063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:23.932824 22999 server_base.cc:1034] running on GCE node
I20260809 16:32:23.933059 22999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:23.933501 22999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:23.934715 22999 hybrid_clock.cc:648] HybridClock initialized: now 1786293143934697 us; error 38 us; skew 500 ppm
Aug 09 16:32:23 dist-test-slave-j2vl krb5kdc[21101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293143, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:23.938381 22999 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:32:23.938966 22999 webserver.cc:460] Webserver started at http://127.30.23.129:37311/ using document root <none> and password file <none>
I20260809 16:32:23.939265 22999 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:23.939335 22999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:23.941193 22999 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:32:23.942163 23077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:23.942418 22999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:32:23.942523 22999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "0b8ed3a2424f40d9929e7f62be54ecbf"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "43bf4d71816d5c6186e23449ecd7e3de"
server_key_iv: "e8da4a38acc49f2aa391d8da782cc21f"
server_key_version: "encryptionkey@0"
I20260809 16:32:23.943061 22999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:23.967415 22999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:23.971053 22999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:23.971247 22999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:23.971549 22999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:23.972136 23087 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:32:23.973827 22999 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:32:23.973901 22999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:32:23.973940 22999 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:32:23.974932 22999 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:32:23.974987 22999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:32:23.975085 23087 tablet_bootstrap.cc:492] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf: Bootstrap starting.
I20260809 16:32:23.983201 22999 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:46363
I20260809 16:32:23.983405 23279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:46363 every 8 connection(s)
I20260809 16:32:23.983736 22999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:32:23.986303 23087 log.cc:824] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:23.988487 23087 tablet_bootstrap.cc:492] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:32:23.988788 23087 tablet_bootstrap.cc:492] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf: Bootstrap complete.
I20260809 16:32:23.989828 23087 ts_tablet_manager.cc:1397] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20260809 16:32:23.990921 23087 raft_consensus.cc:348] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:23.991191 23087 raft_consensus.cc:729] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b8ed3a2424f40d9929e7f62be54ecbf, State: Initialized, Role: FOLLOWER
I20260809 16:32:23.991379 23087 consensus_queue.cc:260] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:23.991705 23087 ts_tablet_manager.cc:1428] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 16:32:23.994107 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 22999
I20260809 16:32:23.994536 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:42163
--local_ip_for_outbound_sockets=127.30.23.130
--tserver_master_addrs=127.30.23.190:38345
--webserver_port=42043
--webserver_interface=127.30.23.130
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:24.064932 23282 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:32:23.984686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:39769)
0809 16:32:23.984968 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:23.985024 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:23.985897 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:24.064670 (+ 78773us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":127,"threads_started":1}
W20260809 16:32:24.065246 23280 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:38345 (0 consecutive failures): Network error: Failed to ping master at 127.30.23.190:38345: Client connection negotiation failed: client connection to 127.30.23.190:38345: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:32:24.065636 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:24.065511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:44975)
0809 16:32:24.065568 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:24.065608 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 16:32:24.066061 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:24.065921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:34111)
0809 16:32:24.065996 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:24.066037 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 16:32:24.066227 23280 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:32:24.088776 23286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:24.089057 23286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:24.089103 23286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:24.090518 23286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:24.090627 23286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:24.090688 23286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:32:24.092841 23286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:42163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=42043
--tserver_master_addrs=127.30.23.190:38345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:24.093168 23286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:24.093499 23286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:24.099771 23298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:24.099823 23295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:24.099809 23294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:24.100091 23286 server_base.cc:1034] running on GCE node
I20260809 16:32:24.100302 23286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:24.100535 23286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:24.101716 23286 hybrid_clock.cc:648] HybridClock initialized: now 1786293144101704 us; error 36 us; skew 500 ppm
Aug 09 16:32:24 dist-test-slave-j2vl krb5kdc[21101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293144, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:24.104496 23286 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:32:24.105005 23286 webserver.cc:460] Webserver started at http://127.30.23.130:42043/ using document root <none> and password file <none>
I20260809 16:32:24.105268 23286 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:24.105342 23286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:24.106865 23286 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:32:24.107684 23306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:24.108057 23286 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:32:24.108177 23286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "b275b0d7161a4602a97533b6445cf042"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "642fc3068f5ba762b088aa71bf0b43c8"
server_key_iv: "474022df76c511b08cfd5a6fe422abad"
server_key_version: "encryptionkey@0"
I20260809 16:32:24.108595 23286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:24.131775 23286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:24.134409 23286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:24.134594 23286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:24.134874 23286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:24.135210 23286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:24.135268 23286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:24.135315 23286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:24.135347 23286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:24.142388 23286 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:42163
I20260809 16:32:24.142434 23493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:42163 every 8 connection(s)
I20260809 16:32:24.142805 23286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:32:24.143903 23499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:24.143511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:44747)
0809 16:32:24.143677 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:24.143757 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
W20260809 16:32:24.144374 23494 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:38345 (0 consecutive failures): Network error: Failed to ping master at 127.30.23.190:38345: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
I20260809 16:32:24.144699 23499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:24.144610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:59931)
0809 16:32:24.144654 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:24.144681 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 16:32:24.144975 23499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:24.144913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:33319)
0809 16:32:24.144938 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:24.144962 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260809 16:32:24.145110 23494 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:32:24.151640 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 23286
I20260809 16:32:24.151991 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:39977
--local_ip_for_outbound_sockets=127.30.23.131
--tserver_master_addrs=127.30.23.190:38345
--webserver_port=33443
--webserver_interface=127.30.23.131
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:24.239042 23500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:24.239434 23500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:24.239519 23500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:24.240954 23500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:24.241014 23500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:24.241045 23500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:32:24.242838 23500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:39977
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=33443
--tserver_master_addrs=127.30.23.190:38345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:24.243150 23500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:24.243438 23500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:32:24.249498 23500 server_base.cc:1034] running on GCE node
W20260809 16:32:24.249480 23509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:24.249470 23510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:24.249464 23513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:24.249985 23500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:24.250319 23500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:24.251490 23500 hybrid_clock.cc:648] HybridClock initialized: now 1786293144251465 us; error 36 us; skew 500 ppm
Aug 09 16:32:24 dist-test-slave-j2vl krb5kdc[21101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293144, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:24.254292 23500 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:32:24.254860 23500 webserver.cc:460] Webserver started at http://127.30.23.131:33443/ using document root <none> and password file <none>
I20260809 16:32:24.255098 23500 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:24.255170 23500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:24.256597 23500 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:32:24.257388 23526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:24.257624 23500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:32:24.257740 23500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "173bb4832a024ed9ad3cdc56c025e396"
format_stamp: "Formatted at 2026-08-09 16:32:22 on dist-test-slave-j2vl"
server_key: "934da374184e86274f58e8230a4df9f2"
server_key_iv: "ce95de094f85e32bf151ef98c61446ec"
server_key_version: "encryptionkey@0"
I20260809 16:32:24.258149 23500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:24.283514 23500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:24.286187 23500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:24.286381 23500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:24.286661 23500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:24.287125 23500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:24.287191 23500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:24.287230 23500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:24.287281 23500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:24.293447 23500 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:39977
I20260809 16:32:24.293498 23707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:39977 every 8 connection(s)
I20260809 16:32:24.293848 23500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:32:24.294898 23712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:24.294508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:37363)
0809 16:32:24.294667 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:24.294759 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
W20260809 16:32:24.295403 23708 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:38345 (0 consecutive failures): Network error: Failed to ping master at 127.30.23.190:38345: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
I20260809 16:32:24.295706 23712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:24.295629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:35315)
0809 16:32:24.295660 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:24.295688 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260809 16:32:24.296094 23712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:24.296013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:51949)
0809 16:32:24.296057 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:24.296080 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 16:32:24.296252 23708 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:32:24.298556 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 23500
I20260809 16:32:25.067134 23715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:25.066792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:46521)
0809 16:32:25.067001 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:25.067044 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
I20260809 16:32:25.145885 23716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:25.145623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:33221)
0809 16:32:25.145773 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:25.145820 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
I20260809 16:32:25.296875 23717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:25.296653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:42817)
0809 16:32:25.296777 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:25.296815 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260809 16:32:25.451661 23718 raft_consensus.cc:482] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:32:25.451875 23718 raft_consensus.cc:504] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:25.452311 23718 leader_election.cc:288] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977), b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163)
Aug 09 16:32:25 dist-test-slave-j2vl krb5kdc[21101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293143, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.131@KRBTEST.COM
Aug 09 16:32:25 dist-test-slave-j2vl krb5kdc[21101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293143, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.130@KRBTEST.COM
I20260809 16:32:25.464183 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:25.452655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:54953 (local address 127.30.23.131:39977)
0809 16:32:25.452861 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:25.452874 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:25.452912 (+ 38us) server_negotiation.cc:407] Connection header received
0809 16:32:25.453466 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:25.453469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:25.453529 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:25.453688 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:25.454472 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:25.455337 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:25.456367 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:25.456822 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:25.456938 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:25.459606 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:25.459612 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:25.459613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:25.459628 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:25.462076 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:25.462706 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:25.462710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:25.462710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:25.462813 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:25.463148 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:25.463149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:25.463149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:25.463496 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:25.463600 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:25.463896 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 16:32:25.464067 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":83,"threads_started":1}
I20260809 16:32:25.464169 23720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:25.452670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:42163 (local address 127.30.23.129:52113)
0809 16:32:25.452840 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:25.452850 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:25.452937 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:25.453739 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:25.453740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:25.454137 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:25.454332 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:25.454338 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:25.455461 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:25.455462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:25.456270 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:25.456272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:25.456876 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:25.456877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:25.457009 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:25.457698 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:25.457707 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:25.459734 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:25.462482 (+ 2748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:25.462483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:25.462483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:25.462683 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:25.462985 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:25.462986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:25.462986 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:25.463071 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:25.463580 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:25.463588 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:25.463719 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 16:32:25.463837 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 16:32:25.464060 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":83,"thread_start_us":61,"threads_started":1}
I20260809 16:32:25.464193 23722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:25.452846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:52113 (local address 127.30.23.130:42163)
0809 16:32:25.452998 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:25.453009 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:25.453047 (+ 38us) server_negotiation.cc:407] Connection header received
0809 16:32:25.453547 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:25.453549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:25.453589 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:25.453734 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:25.454456 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:25.455326 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:25.456378 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:25.456781 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:25.456905 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:25.459867 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:25.459871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:25.459872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:25.459884 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:25.462377 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:25.462790 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:25.462793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:25.462794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:25.462912 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:25.463155 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:25.463157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:25.463157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:25.463497 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:25.463601 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:25.463900 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 16:32:25.464052 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260809 16:32:25.464283 23715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:25.452527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:39977 (local address 127.30.23.129:54953)
0809 16:32:25.452588 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:25.452601 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:25.452661 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:25.453733 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:25.453735 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:25.454136 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:25.454332 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:25.454338 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:25.455449 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:25.455450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:25.456269 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:25.456271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:25.456904 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:25.456906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:25.457009 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:25.457758 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:25.457763 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:25.459410 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:25.462195 (+ 2785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:25.462206 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:25.462209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:25.462593 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:25.462920 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:25.462920 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:25.462921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:25.463071 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:25.463581 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:25.463588 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:25.463719 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 16:32:25.463837 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 16:32:25.464145 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":138}
W20260809 16:32:25.465116 23082 leader_election.cc:341] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163): Not found: Tablet not found: 353ee0fdab9f4f888d6856e58f6d01c5
W20260809 16:32:25.465221 23082 leader_election.cc:341] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977): Not found: Tablet not found: 353ee0fdab9f4f888d6856e58f6d01c5
I20260809 16:32:25.465272 23082 leader_election.cc:302] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b8ed3a2424f40d9929e7f62be54ecbf; no voters: 173bb4832a024ed9ad3cdc56c025e396, b275b0d7161a4602a97533b6445cf042
I20260809 16:32:25.465411 23718 raft_consensus.cc:2726] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:32:26.068477 23723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:26.068162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:57771)
0809 16:32:26.068371 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:26.068427 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":84,"threads_started":1}
I20260809 16:32:26.147125 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:26.146834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:56291)
0809 16:32:26.147031 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:26.147079 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":142,"threads_started":1}
I20260809 16:32:26.297566 23726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:26.297313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:32769)
0809 16:32:26.297495 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:26.297528 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":112,"threads_started":1}
I20260809 16:32:27.069716 23727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:27.069449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:59629)
0809 16:32:27.069624 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:27.069665 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":114,"threads_started":1}
I20260809 16:32:27.147967 23728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:27.147710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:45349)
0809 16:32:27.147871 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:27.147913 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260809 16:32:27.298328 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:27.298072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:50669)
0809 16:32:27.298246 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:27.298287 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":72,"threads_started":1}
I20260809 16:32:27.638432 23730 raft_consensus.cc:482] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:32:27.638538 23730 raft_consensus.cc:504] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:27.638753 23730 leader_election.cc:288] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977), b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163)
W20260809 16:32:27.639325 23082 leader_election.cc:341] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977): Not found: Tablet not found: 353ee0fdab9f4f888d6856e58f6d01c5
W20260809 16:32:27.639406 23082 leader_election.cc:341] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163): Not found: Tablet not found: 353ee0fdab9f4f888d6856e58f6d01c5
I20260809 16:32:27.639424 23082 leader_election.cc:302] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b8ed3a2424f40d9929e7f62be54ecbf; no voters: 173bb4832a024ed9ad3cdc56c025e396, b275b0d7161a4602a97533b6445cf042
I20260809 16:32:27.639508 23730 raft_consensus.cc:2726] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:32:28.070721 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:28.070436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:46743)
0809 16:32:28.070610 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:28.070670 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":71,"threads_started":1}
I20260809 16:32:28.148777 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:28.148537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:41811)
0809 16:32:28.148676 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:28.148723 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
I20260809 16:32:28.299044 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:28.298789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:37853)
0809 16:32:28.298961 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:28.299001 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":84,"threads_started":1}
I20260809 16:32:29.071620 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:29.071364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:56153)
0809 16:32:29.071527 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:29.071571 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
I20260809 16:32:29.150511 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:29.149775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:41659)
0809 16:32:29.150137 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:29.150357 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":221,"threads_started":1}
I20260809 16:32:29.299871 23740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:29.299639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:35147)
0809 16:32:29.299781 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:29.299825 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260809 16:32:29.558662 23744 raft_consensus.cc:482] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:32:29.558770 23744 raft_consensus.cc:504] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:29.558985 23744 leader_election.cc:288] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977), b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163)
W20260809 16:32:29.559572 23082 leader_election.cc:341] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977): Not found: Tablet not found: 353ee0fdab9f4f888d6856e58f6d01c5
W20260809 16:32:29.559681 23082 leader_election.cc:341] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163): Not found: Tablet not found: 353ee0fdab9f4f888d6856e58f6d01c5
I20260809 16:32:29.559762 23082 leader_election.cc:302] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b8ed3a2424f40d9929e7f62be54ecbf; no voters: 173bb4832a024ed9ad3cdc56c025e396, b275b0d7161a4602a97533b6445cf042
I20260809 16:32:29.559870 23744 raft_consensus.cc:2726] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:32:30.072485 23745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:30.072201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:37913)
0809 16:32:30.072371 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:30.072420 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":70,"threads_started":1}
I20260809 16:32:30.151515 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:30.151292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:50419)
0809 16:32:30.151428 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:30.151470 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
I20260809 16:32:30.300688 23747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:30.300429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:51031)
0809 16:32:30.300608 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:30.300648 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":93,"threads_started":1}
I20260809 16:32:31.073326 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:31.073059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:38995)
0809 16:32:31.073222 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:31.073278 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":68,"threads_started":1}
I20260809 16:32:31.152434 23750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:31.152133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:39271)
0809 16:32:31.152330 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:31.152377 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":91,"threads_started":1}
I20260809 16:32:31.301700 23751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:31.301462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:44591)
0809 16:32:31.301610 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:31.301653 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260809 16:32:32.074211 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:32.073942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:38291)
0809 16:32:32.074125 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:32.074161 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":71,"threads_started":1}
I20260809 16:32:32.153520 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:32.153174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:39813)
0809 16:32:32.153415 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:32.153472 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":79,"threads_started":1}
I20260809 16:32:32.302469 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:32.302209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:38939)
0809 16:32:32.302377 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:32.302420 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":69,"threads_started":1}
I20260809 16:32:33.075217 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:33.074856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:50301)
0809 16:32:33.075073 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:33.075128 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":71,"threads_started":1}
I20260809 16:32:33.154337 23759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:33.154063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:45393)
0809 16:32:33.154231 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:33.154278 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260809 16:32:33.303188 23760 raft_consensus.cc:482] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:32:33.303344 23760 raft_consensus.cc:504] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:33.303467 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:33.303173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:57855)
0809 16:32:33.303289 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:33.303332 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
I20260809 16:32:33.303577 23760 leader_election.cc:288] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977), b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163)
W20260809 16:32:33.303972 23082 leader_election.cc:341] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977): Not found: Tablet not found: 353ee0fdab9f4f888d6856e58f6d01c5
W20260809 16:32:33.304054 23082 leader_election.cc:341] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163): Not found: Tablet not found: 353ee0fdab9f4f888d6856e58f6d01c5
I20260809 16:32:33.304072 23082 leader_election.cc:302] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b8ed3a2424f40d9929e7f62be54ecbf; no voters: 173bb4832a024ed9ad3cdc56c025e396, b275b0d7161a4602a97533b6445cf042
I20260809 16:32:33.304205 23760 raft_consensus.cc:2726] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:32:34.076056 23764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:34.075788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:37863)
0809 16:32:34.075961 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:34.076006 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
I20260809 16:32:34.155261 23766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:34.154970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:49801)
0809 16:32:34.155163 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:34.155200 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":81,"threads_started":1}
I20260809 16:32:34.304263 23768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:34.304006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:56761)
0809 16:32:34.304174 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:34.304213 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260809 16:32:35.076972 23770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:35.076717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:47211)
0809 16:32:35.076882 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:35.076923 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":84,"threads_started":1}
I20260809 16:32:35.156175 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:35.155915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:51473)
0809 16:32:35.156075 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:35.156127 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":67,"threads_started":1}
I20260809 16:32:35.305126 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:35.304853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:38085)
0809 16:32:35.305025 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:35.305072 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":80,"threads_started":1}
I20260809 16:32:36.078013 23774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:36.077732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:56217)
0809 16:32:36.077911 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:36.077962 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":71,"threads_started":1}
I20260809 16:32:36.156999 23776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:36.156718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:50283)
0809 16:32:36.156889 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:36.156937 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":67,"threads_started":1}
I20260809 16:32:36.305997 23777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:36.305714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:44303)
0809 16:32:36.305893 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:36.305945 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
I20260809 16:32:37.078924 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:37.078658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:53977)
0809 16:32:37.078828 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:37.078874 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":73,"threads_started":1}
I20260809 16:32:37.157933 23780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:37.157650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:46955)
0809 16:32:37.157834 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:37.157886 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":68,"threads_started":1}
I20260809 16:32:37.306895 23782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:37.306634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:49075)
0809 16:32:37.306795 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:37.306845 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
I20260809 16:32:38.079810 23783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:38.079524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:43371)
0809 16:32:38.079686 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:38.079761 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":72,"threads_started":1}
I20260809 16:32:38.158726 23784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:38.158486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:42165)
0809 16:32:38.158635 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:38.158680 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260809 16:32:38.307781 23785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:38.307528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:43945)
0809 16:32:38.307683 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:38.307728 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260809 16:32:39.081051 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:39.080788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.129:60509)
0809 16:32:39.080959 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:39.081004 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":76,"threads_started":1}
I20260809 16:32:39.159654 23788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:39.159408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.130:35989)
0809 16:32:39.159567 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:39.159608 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":75,"threads_started":1}
I20260809 16:32:39.259548 23789 raft_consensus.cc:482] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:32:39.259703 23789 raft_consensus.cc:504] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173bb4832a024ed9ad3cdc56c025e396" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 39977 } } peers { permanent_uuid: "b275b0d7161a4602a97533b6445cf042" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 42163 } } peers { permanent_uuid: "0b8ed3a2424f40d9929e7f62be54ecbf" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46363 } }
I20260809 16:32:39.259958 23789 leader_election.cc:288] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977), b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163)
W20260809 16:32:39.260389 23082 leader_election.cc:341] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b275b0d7161a4602a97533b6445cf042 (127.30.23.130:42163): Not found: Tablet not found: 353ee0fdab9f4f888d6856e58f6d01c5
W20260809 16:32:39.260463 23082 leader_election.cc:341] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 173bb4832a024ed9ad3cdc56c025e396 (127.30.23.131:39977): Not found: Tablet not found: 353ee0fdab9f4f888d6856e58f6d01c5
I20260809 16:32:39.260480 23082 leader_election.cc:302] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b8ed3a2424f40d9929e7f62be54ecbf; no voters: 173bb4832a024ed9ad3cdc56c025e396, b275b0d7161a4602a97533b6445cf042
I20260809 16:32:39.260610 23789 raft_consensus.cc:2726] T 353ee0fdab9f4f888d6856e58f6d01c5 P 0b8ed3a2424f40d9929e7f62be54ecbf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:32:39.300498 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 22999
I20260809 16:32:39.309674 23792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:39.308420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:38345 (local address 127.30.23.131:56509)
0809 16:32:39.309578 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:39.309615 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:38345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1136,"thread_start_us":1084,"threads_started":1}
I20260809 16:32:39.315820 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 23286
I20260809 16:32:39.330751 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 23500
2026-08-09T16:32:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18752 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:32:39 dist-test-slave-j2vl krb5kdc[23797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:32:39 dist-test-slave-j2vl krb5kdc[23797](info): set up 2 sockets
Aug 09 16:32:39 dist-test-slave-j2vl krb5kdc[23797](info): commencing operation
krb5kdc: starting...
W20260809 16:32:40.637198 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:32:40 dist-test-slave-j2vl krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:32:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:32:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:32:40.790719 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:33381
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:33381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:40.883569 23818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:40.883826 23818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:40.883854 23818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:40.885195 23818 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 16:32:40.885252 23818 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 16:32:40.885273 23818 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 16:32:40.885285 23818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:32:40.885299 23818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:40.885314 23818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:40.885326 23818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:32:40.885411 23818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:32:40.887243 23818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:33381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:33381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:40.887581 23818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:40.887873 23818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:40.893640 23823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:40.893668 23826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:40.893795 23818 server_base.cc:1034] running on GCE node
W20260809 16:32:40.893636 23831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:40.894402 23818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:40.894797 23818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:40.896010 23818 hybrid_clock.cc:648] HybridClock initialized: now 1786293160895974 us; error 56 us; skew 500 ppm
Aug 09 16:32:40 dist-test-slave-j2vl krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:40.899060 23818 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:32:40.899555 23818 webserver.cc:460] Webserver started at http://127.30.23.190:46495/ using document root <none> and password file <none>
I20260809 16:32:40.899838 23818 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:40.899896 23818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:40.900000 23818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:32:40.901192 23818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "d0bfbed6462a44a4aee381609e12ca24"
format_stamp: "Formatted at 2026-08-09 16:32:40 on dist-test-slave-j2vl"
server_key: "234518f36f970b678d4632a0052aa159"
server_key_iv: "ac5b8f3ec09bcfa1d33d8faaa0e70c3a"
server_key_version: "encryptionkey@0"
I20260809 16:32:40.901664 23818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "d0bfbed6462a44a4aee381609e12ca24"
format_stamp: "Formatted at 2026-08-09 16:32:40 on dist-test-slave-j2vl"
server_key: "234518f36f970b678d4632a0052aa159"
server_key_iv: "ac5b8f3ec09bcfa1d33d8faaa0e70c3a"
server_key_version: "encryptionkey@0"
I20260809 16:32:40.903541 23818 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:32:40.904858 23843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:40.905237 23818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:40.905419 23818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "d0bfbed6462a44a4aee381609e12ca24"
format_stamp: "Formatted at 2026-08-09 16:32:40 on dist-test-slave-j2vl"
server_key: "234518f36f970b678d4632a0052aa159"
server_key_iv: "ac5b8f3ec09bcfa1d33d8faaa0e70c3a"
server_key_version: "encryptionkey@0"
I20260809 16:32:40.905500 23818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:40.934216 23818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:40.936887 23818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:40.937089 23818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:40.942584 23818 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:33381
I20260809 16:32:40.942667 23932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:33381 every 8 connection(s)
I20260809 16:32:40.943097 23818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:32:40.944180 23934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:40.946653 23934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Bootstrap starting.
I20260809 16:32:40.947338 23934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:40.947727 23934 log.cc:824] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:40.947830 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 23818
I20260809 16:32:40.947952 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:32:40.948164 30814 external_mini_cluster.cc:1428] Setting key 096f32d945bd214da76c188a2f008b73
I20260809 16:32:40.948683 23934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: No bootstrap required, opened a new log
I20260809 16:32:40.950605 23934 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } }
I20260809 16:32:40.950791 23934 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:40.950858 23934 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0bfbed6462a44a4aee381609e12ca24, State: Initialized, Role: FOLLOWER
I20260809 16:32:40.950999 23934 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } }
I20260809 16:32:40.951085 23934 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:32:40.951113 23934 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:32:40.951157 23934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:40.952149 23934 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } }
I20260809 16:32:40.952293 23934 leader_election.cc:302] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0bfbed6462a44a4aee381609e12ca24; no voters:
I20260809 16:32:40.952488 23934 leader_election.cc:288] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:32:40.952556 23942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:32:40.952703 23942 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 1 LEADER]: Becoming Leader. State: Replica: d0bfbed6462a44a4aee381609e12ca24, State: Running, Role: LEADER
I20260809 16:32:40.952783 23934 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:32:40.952845 23942 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } }
I20260809 16:32:40.953410 23942 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0bfbed6462a44a4aee381609e12ca24. Latest consensus state: current_term: 1 leader_uuid: "d0bfbed6462a44a4aee381609e12ca24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } } }
I20260809 16:32:40.953491 23942 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:40.953717 23953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:32:40.954267 23953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:32:40.954615 23944 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0bfbed6462a44a4aee381609e12ca24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } } }
I20260809 16:32:40.954878 23944 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: This master's current role is: LEADER
I20260809 16:32:40.956468 23953 catalog_manager.cc:1347] Generated new cluster ID: b419b61700ab41e1b5f160c1a503bfd1
I20260809 16:32:40.956524 23953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:32:40 dist-test-slave-j2vl krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:40.962575 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:40.948887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53696 (local address 127.30.23.190:33381)
0809 16:32:40.949075 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:40.949092 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:40.949127 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:32:40.950602 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:40.950612 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:40.950674 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:40.950863 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:40.951683 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:40.952993 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:40.954292 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:40.954989 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:40.955107 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:40.958075 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:40.958079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:40.958080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:40.958093 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:40.960184 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:40.960624 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:40.960629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:40.960629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:40.960778 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:40.961010 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:40.961011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:40.961011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:40.961383 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:40.961550 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:40.962169 (+ 619us) server_negotiation.cc:299] Negotiation successful
0809 16:32:40.962355 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260809 16:32:40.962695 23953 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:32:40.963899 23953 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:32:40.969156 23953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Generated new TSK 0
I20260809 16:32:40.969451 23953 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:32:41.009919 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33381
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:41.130182 23974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:41.130462 23974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:41.130507 23974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:41.131778 23974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:41.131861 23974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:41.131911 23974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:32:41.133615 23974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:41.133924 23974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:41.134197 23974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:41.140347 23980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:41.140354 23988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:41.140549 23982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:41.140848 23974 server_base.cc:1034] running on GCE node
I20260809 16:32:41.141120 23974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:41.141455 23974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:41.142630 23974 hybrid_clock.cc:648] HybridClock initialized: now 1786293161142616 us; error 29 us; skew 500 ppm
Aug 09 16:32:41 dist-test-slave-j2vl krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293161, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:41.145581 23974 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:32:41.146131 23974 webserver.cc:460] Webserver started at http://127.30.23.129:36257/ using document root <none> and password file <none>
I20260809 16:32:41.146418 23974 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:41.146494 23974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:41.146663 23974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:32:41.147780 23974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "98cdb8a0a78c4179b6421b96d1658f9d"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "671e76e497dc3a28d81a3d936135bc87"
server_key_iv: "b25ffb7a91acaa1576aca80f2d52d50d"
server_key_version: "encryptionkey@0"
I20260809 16:32:41.148176 23974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "98cdb8a0a78c4179b6421b96d1658f9d"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "671e76e497dc3a28d81a3d936135bc87"
server_key_iv: "b25ffb7a91acaa1576aca80f2d52d50d"
server_key_version: "encryptionkey@0"
I20260809 16:32:41.149802 23974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:32:41.150714 24001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:41.150954 23974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:41.151079 23974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "98cdb8a0a78c4179b6421b96d1658f9d"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "671e76e497dc3a28d81a3d936135bc87"
server_key_iv: "b25ffb7a91acaa1576aca80f2d52d50d"
server_key_version: "encryptionkey@0"
I20260809 16:32:41.151168 23974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:41.175882 23974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:41.178637 23974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:41.178886 23974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:41.179175 23974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:41.179533 23974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:41.179593 23974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:41.179633 23974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:41.179677 23974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:41.185946 23974 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:45881
I20260809 16:32:41.185977 24191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:45881 every 8 connection(s)
I20260809 16:32:41.186383 23974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:32:41.187667 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 23974
I20260809 16:32:41.187810 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:32:41.188019 30814 external_mini_cluster.cc:1428] Setting key 4d345ccebdf61002f23017b94b1f96ad
Aug 09 16:32:41 dist-test-slave-j2vl krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293161, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
I20260809 16:32:41.197832 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.187209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:47607 (local address 127.30.23.190:33381)
0809 16:32:41.187311 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:41.187314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:41.187511 (+ 197us) server_negotiation.cc:407] Connection header received
0809 16:32:41.188268 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:41.188270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:41.188290 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:41.188352 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:41.189572 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.190345 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.191455 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.191687 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.191786 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.194584 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:41.194586 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:41.194586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:41.194604 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:41.196197 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:41.196695 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:41.196697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:41.196697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:41.196762 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:41.197045 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:41.197046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:41.197046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:41.197181 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:41.197263 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:41.197618 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 16:32:41.197731 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 16:32:41.198081 24196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.187141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:47607)
0809 16:32:41.187337 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:41.187376 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:41.188136 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:41.188459 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:41.188464 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:41.189166 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:41.189464 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.189470 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.190487 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.190488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.191345 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.191348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.191791 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.191792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.191951 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.192585 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:41.192597 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:41.194454 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:41.196316 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:41.196326 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:41.196329 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:41.196598 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:41.196846 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:41.196847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:41.196847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:41.196969 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:41.197266 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:41.197275 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:41.197471 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 16:32:41.197593 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 16:32:41.197769 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":122,"threads_started":1}
I20260809 16:32:41.198748 24192 heartbeater.cc:347] Connected to a master server at 127.30.23.190:33381
I20260809 16:32:41.198845 24192 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:41.199071 24192 heartbeater.cc:511] Master 127.30.23.190:33381 requested a full tablet report, sending...
I20260809 16:32:41.199702 23874 ts_manager.cc:194] Registered new tserver with Master: 98cdb8a0a78c4179b6421b96d1658f9d (127.30.23.129:45881)
I20260809 16:32:41.200416 23874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:47607
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:32:41.222021 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33381
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:41.312981 24202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:41.313210 24202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:41.313237 24202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:41.314478 24202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:41.314527 24202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:41.314555 24202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:32:41.316054 24202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:41.316275 24202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:41.316468 24202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:41.322084 24211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:41.322067 24212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:41.322263 24202 server_base.cc:1034] running on GCE node
W20260809 16:32:41.322067 24216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:41.322602 24202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:41.322897 24202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:41.324090 24202 hybrid_clock.cc:648] HybridClock initialized: now 1786293161324085 us; error 40 us; skew 500 ppm
Aug 09 16:32:41 dist-test-slave-j2vl krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293161, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:41.326897 24202 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:32:41.327440 24202 webserver.cc:460] Webserver started at http://127.30.23.130:35585/ using document root <none> and password file <none>
I20260809 16:32:41.327692 24202 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:41.327762 24202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:41.327915 24202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:32:41.328992 24202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "be085fa6b245c786ca8aacbc1100e21d"
server_key_iv: "000b1cdcbc46fdd2ffc9299ff56ec997"
server_key_version: "encryptionkey@0"
I20260809 16:32:41.329445 24202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "be085fa6b245c786ca8aacbc1100e21d"
server_key_iv: "000b1cdcbc46fdd2ffc9299ff56ec997"
server_key_version: "encryptionkey@0"
I20260809 16:32:41.330931 24202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:41.331835 24227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:41.332086 24202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:41.332206 24202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "be085fa6b245c786ca8aacbc1100e21d"
server_key_iv: "000b1cdcbc46fdd2ffc9299ff56ec997"
server_key_version: "encryptionkey@0"
I20260809 16:32:41.332315 24202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:41.347453 24202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:41.350046 24202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:41.350231 24202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:41.350490 24202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:41.350835 24202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:41.350898 24202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:41.350941 24202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:41.350971 24202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:41.358031 24202 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:36689
I20260809 16:32:41.358081 24409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:36689 every 8 connection(s)
I20260809 16:32:41.358508 24202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:32:41.359117 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 24202
I20260809 16:32:41.359236 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:32:41.359484 30814 external_mini_cluster.cc:1428] Setting key 9422758c986fedace0a086963b2ac837
Aug 09 16:32:41 dist-test-slave-j2vl krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293161, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
I20260809 16:32:41.370850 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.359438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:46785 (local address 127.30.23.190:33381)
0809 16:32:41.359521 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:41.359524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:41.359744 (+ 220us) server_negotiation.cc:407] Connection header received
0809 16:32:41.360452 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:41.360453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:41.360464 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:41.360533 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:41.361952 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.362529 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.364024 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.364372 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.364459 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.367022 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:41.367024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:41.367024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:41.367034 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:41.369024 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:41.369666 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:41.369669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:41.369669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:41.369740 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:41.370092 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:41.370093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:41.370094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:41.370277 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:41.370352 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:41.370695 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 16:32:41.370781 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:32:41.371031 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.359374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:46785)
0809 16:32:41.359591 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:41.359649 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:41.360316 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:41.360616 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:41.360621 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:41.361537 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:41.361836 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.361843 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.362637 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.362638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.363914 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.363917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.364483 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.364484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.364588 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.365197 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:41.365210 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:41.366913 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:41.369152 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:41.369161 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:41.369163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:41.369568 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:41.369863 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:41.369864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:41.369864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:41.369989 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:41.370345 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:41.370353 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:41.370506 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 16:32:41.370602 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 16:32:41.370747 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":117,"threads_started":1}
I20260809 16:32:41.371827 24410 heartbeater.cc:347] Connected to a master server at 127.30.23.190:33381
I20260809 16:32:41.371927 24410 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:41.372133 24410 heartbeater.cc:511] Master 127.30.23.190:33381 requested a full tablet report, sending...
I20260809 16:32:41.372695 23874 ts_manager.cc:194] Registered new tserver with Master: f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689)
I20260809 16:32:41.373117 23874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:46785
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:32:41.394034 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33381
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:32:41.483361 24418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:32:41.483604 24418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:32:41.483632 24418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:32:41.484917 24418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:32:41.484969 24418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:32:41.484997 24418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:32:41.486649 24418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:32:41.486922 24418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:32:41.487186 24418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:32:41.492699 24427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:41.492756 24432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:32:41.492792 24426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:32:41.493534 24418 server_base.cc:1034] running on GCE node
I20260809 16:32:41.493767 24418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:32:41.494031 24418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:32:41.495241 24418 hybrid_clock.cc:648] HybridClock initialized: now 1786293161495188 us; error 69 us; skew 500 ppm
Aug 09 16:32:41 dist-test-slave-j2vl krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293161, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:32:41.498343 24418 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:32:41.498845 24418 webserver.cc:460] Webserver started at http://127.30.23.131:41579/ using document root <none> and password file <none>
I20260809 16:32:41.499099 24418 fs_manager.cc:359] Metadata directory not provided
I20260809 16:32:41.499173 24418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:32:41.499326 24418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:32:41.500464 24418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "dc78da247c0438e66c96898b66d64ea3"
server_key_iv: "7e154dd1c185176431307ba7dc321982"
server_key_version: "encryptionkey@0"
I20260809 16:32:41.500854 24418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "dc78da247c0438e66c96898b66d64ea3"
server_key_iv: "7e154dd1c185176431307ba7dc321982"
server_key_version: "encryptionkey@0"
I20260809 16:32:41.502416 24418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:32:41.503508 24442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:41.503858 24418 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:32:41.504024 24418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "dc78da247c0438e66c96898b66d64ea3"
server_key_iv: "7e154dd1c185176431307ba7dc321982"
server_key_version: "encryptionkey@0"
I20260809 16:32:41.504130 24418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:32:41.516444 24418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:32:41.519465 24418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:32:41.519629 24418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:32:41.519887 24418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:32:41.520236 24418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:32:41.520298 24418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:41.520345 24418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:32:41.520537 24418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:32:41.527947 24418 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:41333
I20260809 16:32:41.528005 24628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:41333 every 8 connection(s)
I20260809 16:32:41.528400 24418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:32:41.530586 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 24418
I20260809 16:32:41.530718 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:32:41.530903 30814 external_mini_cluster.cc:1428] Setting key f652f00e562e12cc46bca3a14cfc6489
Aug 09 16:32:41 dist-test-slave-j2vl krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293161, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:32:41.539458 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.529570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:41089 (local address 127.30.23.190:33381)
0809 16:32:41.529630 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:41.529633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:41.529933 (+ 300us) server_negotiation.cc:407] Connection header received
0809 16:32:41.530409 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:41.530411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:41.530428 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:41.530513 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:41.531633 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.532193 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.533224 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.533534 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.533603 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.536292 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:41.536293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:41.536294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:41.536301 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:41.537815 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:41.538301 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:41.538302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:41.538302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:41.538364 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:41.538705 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:41.538707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:41.538707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:41.538861 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:41.538940 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:41.539294 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 16:32:41.539377 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:32:41.539605 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.529531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:41089)
0809 16:32:41.529726 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:41.529765 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:41.530275 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:41.530580 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:41.530584 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:41.531331 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:32:41.531537 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.531541 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.532290 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.532291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.533124 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.533126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.533618 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.533619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.533725 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.534296 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:32:41.534305 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:32:41.536096 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:32:41.537924 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:41.537931 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:41.537934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:41.538206 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:41.538472 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:32:41.538473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:32:41.538473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:32:41.538597 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:32:41.538977 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:32:41.538988 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:32:41.539128 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 16:32:41.539232 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 16:32:41.539389 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260809 16:32:41.540230 24630 heartbeater.cc:347] Connected to a master server at 127.30.23.190:33381
I20260809 16:32:41.540331 24630 heartbeater.cc:464] Registering TS with master...
I20260809 16:32:41.540490 24630 heartbeater.cc:511] Master 127.30.23.190:33381 requested a full tablet report, sending...
I20260809 16:32:41.540933 23874 ts_manager.cc:194] Registered new tserver with Master: a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333)
I20260809 16:32:41.541323 23874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:41089
I20260809 16:32:41.542632 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:32:41.550611 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.544181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53702 (local address 127.30.23.190:33381)
0809 16:32:41.544253 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:41.544255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:41.544287 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:32:41.544334 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:41.544335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:41.544343 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:41.544403 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:32:41.544974 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.545572 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.546312 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.546502 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.546559 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.547776 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:32:41.547777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:32:41.547777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:32:41.547784 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:32:41.549291 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:41.549701 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:41.549702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:41.549702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:41.549759 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:32:41.549937 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:32:41.549938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:32:41.549938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:32:41.550128 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:32:41.550230 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:41.550408 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 16:32:41.550499 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 16:32:41.552913 23874 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:32:41.554958 23874 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:32:41.562081 24655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.557853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57332 (local address 127.30.23.129:45881)
0809 16:32:41.558003 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:41.558007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:41.558213 (+ 206us) server_negotiation.cc:407] Connection header received
0809 16:32:41.558256 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:41.558258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:41.558275 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:41.558360 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:41.558749 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.559804 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.561081 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.561642 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.561738 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.561799 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:41.561946 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 16:32:41.561985 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260809 16:32:41.562125 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.557527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:41333 (local address 127.0.0.1:39224)
0809 16:32:41.557945 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:41.557956 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:41.558021 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:41.558370 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:41.558373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:41.558375 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:41.558460 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.558462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.559803 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.559804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.560787 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.560789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.561826 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.561826 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.561919 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.561927 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 16:32:41.561982 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:32:41.562029 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":61,"thread_start_us":93,"threads_started":1}
I20260809 16:32:41.562148 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.557637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43566 (local address 127.30.23.130:36689)
0809 16:32:41.557898 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:41.557904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:41.558017 (+ 113us) server_negotiation.cc:407] Connection header received
0809 16:32:41.558120 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:41.558122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:41.558136 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:41.558249 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:41.558642 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.559406 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.560871 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.561649 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.561726 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.561784 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:41.562027 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 16:32:41.562074 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":112,"threads_started":1}
I20260809 16:32:41.562422 24652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.557642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39224 (local address 127.30.23.131:41333)
0809 16:32:41.557896 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:41.557901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:41.558110 (+ 209us) server_negotiation.cc:407] Connection header received
0809 16:32:41.558158 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:41.558161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:41.558176 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:41.558247 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:41.558550 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.559684 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.560902 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.561649 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.561723 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.561785 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:41.562251 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 16:32:41.562300 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":136,"threads_started":1}
I20260809 16:32:41.562135 24646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.557736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45881 (local address 127.0.0.1:57332)
0809 16:32:41.558130 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:41.558139 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:41.558187 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:41.558454 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:41.558455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:41.558457 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:41.558600 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.558602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.559915 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.559916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.560985 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.560987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.561740 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.561741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.561826 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.561841 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:32:41.561918 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:32:41.561981 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":68,"threads_started":1}
I20260809 16:32:41.563161 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.557522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:36689 (local address 127.0.0.1:43566)
0809 16:32:41.557890 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:41.557938 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:41.558047 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:41.558407 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:41.558408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:41.558411 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:41.558465 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.558467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.559520 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.559521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.560768 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.560771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.561824 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.561825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.561919 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.561929 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 16:32:41.562410 (+ 481us) client_negotiation.cc:240] Negotiation successful
0809 16:32:41.562521 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":4,"thread_start_us":58,"threads_started":1}
I20260809 16:32:41.563251 24089 tablet_service.cc:1467] Processing CreateTablet for tablet fbbb01542bdb4a67b0fcb55b866f9f1a (DEFAULT_TABLE table=test-table [id=e0c2d08b38cc4eb6aab8f95d762b17d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:32:41.563201 24519 tablet_service.cc:1467] Processing CreateTablet for tablet fbbb01542bdb4a67b0fcb55b866f9f1a (DEFAULT_TABLE table=test-table [id=e0c2d08b38cc4eb6aab8f95d762b17d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:32:41.563795 24089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbbb01542bdb4a67b0fcb55b866f9f1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:41.563817 24519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbbb01542bdb4a67b0fcb55b866f9f1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:41.563844 24304 tablet_service.cc:1467] Processing CreateTablet for tablet fbbb01542bdb4a67b0fcb55b866f9f1a (DEFAULT_TABLE table=test-table [id=e0c2d08b38cc4eb6aab8f95d762b17d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:32:41.564255 24304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbbb01542bdb4a67b0fcb55b866f9f1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:32:41.567443 24656 tablet_bootstrap.cc:492] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694: Bootstrap starting.
I20260809 16:32:41.567849 24657 tablet_bootstrap.cc:492] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Bootstrap starting.
I20260809 16:32:41.568264 24656 tablet_bootstrap.cc:654] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:41.568528 24657 tablet_bootstrap.cc:654] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:41.568908 24656 log.cc:824] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:41.569092 24657 log.cc:824] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:41.569551 24658 tablet_bootstrap.cc:492] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Bootstrap starting.
I20260809 16:32:41.569864 24656 tablet_bootstrap.cc:492] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694: No bootstrap required, opened a new log
I20260809 16:32:41.569926 24656 ts_tablet_manager.cc:1397] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:32:41.570252 24658 tablet_bootstrap.cc:654] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Neither blocks nor log segments found. Creating new log.
I20260809 16:32:41.570761 24658 log.cc:824] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Log is configured to *not* fsync() on all Append() calls
I20260809 16:32:41.571586 24657 tablet_bootstrap.cc:492] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: No bootstrap required, opened a new log
I20260809 16:32:41.571596 24658 tablet_bootstrap.cc:492] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0: No bootstrap required, opened a new log
I20260809 16:32:41.571671 24658 ts_tablet_manager.cc:1397] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:32:41.571671 24657 ts_tablet_manager.cc:1397] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260809 16:32:41.571904 24656 raft_consensus.cc:348] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:32:41.572091 24656 raft_consensus.cc:374] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:41.572141 24656 raft_consensus.cc:729] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f07e6d7cde2d40c2bcedd2c2b4cc5694, State: Initialized, Role: FOLLOWER
I20260809 16:32:41.572279 24656 consensus_queue.cc:260] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:32:41.572649 24410 heartbeater.cc:503] Master 127.30.23.190:33381 was elected leader, sending a full tablet report...
I20260809 16:32:41.572968 24656 ts_tablet_manager.cc:1428] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:32:41.573932 24658 raft_consensus.cc:348] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:32:41.573933 24657 raft_consensus.cc:348] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:32:41.574129 24657 raft_consensus.cc:374] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:41.574129 24658 raft_consensus.cc:374] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:32:41.574182 24658 raft_consensus.cc:729] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6267fa1a6464ae6a3b1c47f7fbcc4d0, State: Initialized, Role: FOLLOWER
I20260809 16:32:41.574182 24657 raft_consensus.cc:729] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98cdb8a0a78c4179b6421b96d1658f9d, State: Initialized, Role: FOLLOWER
I20260809 16:32:41.574316 24658 consensus_queue.cc:260] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:32:41.574316 24657 consensus_queue.cc:260] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:32:41.574630 24657 ts_tablet_manager.cc:1428] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 16:32:41.574630 24658 ts_tablet_manager.cc:1428] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:32:41.574806 24192 heartbeater.cc:503] Master 127.30.23.190:33381 was elected leader, sending a full tablet report...
I20260809 16:32:41.575100 24630 heartbeater.cc:503] Master 127.30.23.190:33381 was elected leader, sending a full tablet report...
W20260809 16:32:41.609578 24412 tablet.cc:2369] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:32:41.687250 24194 tablet.cc:2369] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:32:41.697103 24666 raft_consensus.cc:482] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:32:41.697259 24666 raft_consensus.cc:504] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:32:41.697659 24666 leader_election.cc:288] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689), 98cdb8a0a78c4179b6421b96d1658f9d (127.30.23.129:45881)
I20260809 16:32:41.702234 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.698114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:33607 (local address 127.30.23.130:36689)
0809 16:32:41.698200 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:41.698206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:41.698245 (+ 39us) server_negotiation.cc:407] Connection header received
0809 16:32:41.698279 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:41.698280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:41.698291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:41.698374 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:41.698668 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.699462 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.701050 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.701753 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.701864 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.701878 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:41.702109 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 16:32:41.702159 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 16:32:41.702234 24655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.698300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:47825 (local address 127.30.23.129:45881)
0809 16:32:41.698353 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:41.698355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:41.698364 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:32:41.698395 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:41.698396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:41.698407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:41.698456 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:32:41.698668 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.699457 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.701050 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.701725 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.701831 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.701845 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:41.702110 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 16:32:41.702159 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:32:41.702219 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.697952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:36689 (local address 127.30.23.131:33607)
0809 16:32:41.698028 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:41.698041 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:41.698091 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:41.698407 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:41.698408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:41.698410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:41.698489 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.698491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.699589 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.699589 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.700892 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.700896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.701892 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.701893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.701980 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.701982 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:32:41.702055 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 16:32:41.702113 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":43}
I20260809 16:32:41.702225 24668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.698092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45881 (local address 127.30.23.131:47825)
0809 16:32:41.698276 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:41.698286 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:32:41.698331 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:32:41.698480 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:32:41.698481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:32:41.698482 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:32:41.698545 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.698546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.699589 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.699589 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.700892 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:32:41.700896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.701881 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.701883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:32:41.701975 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.701980 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:32:41.702055 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:32:41.702113 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":96,"thread_start_us":126,"threads_started":1}
I20260809 16:32:41.702747 24123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbbb01542bdb4a67b0fcb55b866f9f1a" candidate_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" is_pre_election: true
I20260809 16:32:41.702759 24341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbbb01542bdb4a67b0fcb55b866f9f1a" candidate_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" is_pre_election: true
I20260809 16:32:41.702939 24123 raft_consensus.cc:2454] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6267fa1a6464ae6a3b1c47f7fbcc4d0 in term 0.
I20260809 16:32:41.703047 24341 raft_consensus.cc:2454] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6267fa1a6464ae6a3b1c47f7fbcc4d0 in term 0.
I20260809 16:32:41.703212 24451 leader_election.cc:302] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98cdb8a0a78c4179b6421b96d1658f9d, a6267fa1a6464ae6a3b1c47f7fbcc4d0; no voters:
I20260809 16:32:41.703372 24666 raft_consensus.cc:2781] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:32:41.703550 24666 raft_consensus.cc:482] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:32:41.703579 24666 raft_consensus.cc:3037] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:41.704399 24666 raft_consensus.cc:504] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:32:41.704536 24666 leader_election.cc:288] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [CANDIDATE]: Term 1 election: Requested vote from peers f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689), 98cdb8a0a78c4179b6421b96d1658f9d (127.30.23.129:45881)
I20260809 16:32:41.704806 24341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbbb01542bdb4a67b0fcb55b866f9f1a" candidate_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694"
I20260809 16:32:41.704865 24123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbbb01542bdb4a67b0fcb55b866f9f1a" candidate_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98cdb8a0a78c4179b6421b96d1658f9d"
I20260809 16:32:41.704893 24341 raft_consensus.cc:3037] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:41.704986 24123 raft_consensus.cc:3037] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:32:41.706197 24341 raft_consensus.cc:2454] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6267fa1a6464ae6a3b1c47f7fbcc4d0 in term 1.
I20260809 16:32:41.706390 24123 raft_consensus.cc:2454] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6267fa1a6464ae6a3b1c47f7fbcc4d0 in term 1.
I20260809 16:32:41.706447 24451 leader_election.cc:302] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6267fa1a6464ae6a3b1c47f7fbcc4d0, f07e6d7cde2d40c2bcedd2c2b4cc5694; no voters:
I20260809 16:32:41.706560 24666 raft_consensus.cc:2781] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:32:41.706807 24666 raft_consensus.cc:686] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 1 LEADER]: Becoming Leader. State: Replica: a6267fa1a6464ae6a3b1c47f7fbcc4d0, State: Running, Role: LEADER
I20260809 16:32:41.707230 24666 consensus_queue.cc:237] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:32:41.708626 23874 catalog_manager.cc:5495] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131). New cstate: current_term: 1 leader_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } health_report { overall_health: UNKNOWN } } }
I20260809 16:32:41.747747 24652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.744860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39232 (local address 127.30.23.131:41333)
0809 16:32:41.744928 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:32:41.744930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:32:41.744949 (+ 19us) server_negotiation.cc:407] Connection header received
0809 16:32:41.745106 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:32:41.745108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:32:41.745127 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:32:41.745200 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:32:41.745442 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.745950 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.746788 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:32:41.747011 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:32:41.747083 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:32:41.747342 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:32:41.747462 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:32:41.747541 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:32:41.747611 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 16:32:41.747634 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:32:41.749986 24341 raft_consensus.cc:1261] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 1 FOLLOWER]: Refusing update from remote peer a6267fa1a6464ae6a3b1c47f7fbcc4d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:32:41.749987 24123 raft_consensus.cc:1261] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Refusing update from remote peer a6267fa1a6464ae6a3b1c47f7fbcc4d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:32:41.750383 24666 consensus_queue.cc:1035] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:32:41.750590 24669 consensus_queue.cc:1035] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:32:41.753896 24673 mvcc.cc:205] Tried to move back new op lower bound from 7316656790523940864 to 7316656790353838080. Current Snapshot: MvccSnapshot[applied={T|T < 7316656790523940864}]
I20260809 16:32:41.754048 24674 mvcc.cc:205] Tried to move back new op lower bound from 7316656790523940864 to 7316656790353838080. Current Snapshot: MvccSnapshot[applied={T|T < 7316656790523940864}]
I20260809 16:32:41.754112 24675 mvcc.cc:205] Tried to move back new op lower bound from 7316656790523940864 to 7316656790353838080. Current Snapshot: MvccSnapshot[applied={T|T < 7316656790523940864}]
I20260809 16:32:41.756831 23874 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:32:41.756984 23874 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:32:41.758041 23874 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Sending DeleteTablet for 3 replicas of tablet fbbb01542bdb4a67b0fcb55b866f9f1a
I20260809 16:32:41.758514 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 23974
I20260809 16:32:41.758538 24519 tablet_service.cc:1514] Processing DeleteTablet for tablet fbbb01542bdb4a67b0fcb55b866f9f1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:32:41 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:39224
I20260809 16:32:41.758559 24304 tablet_service.cc:1514] Processing DeleteTablet for tablet fbbb01542bdb4a67b0fcb55b866f9f1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:32:41 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:43566
I20260809 16:32:41.774269 24684 tablet_replica.cc:330] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0: stopping tablet replica
I20260809 16:32:41.774479 24684 raft_consensus.cc:2229] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:32:41.774662 24684 raft_consensus.cc:2258] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:32:41.774897 24685 tablet_replica.cc:330] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694: stopping tablet replica
I20260809 16:32:41.775020 24685 raft_consensus.cc:2229] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:32:41.775131 24685 raft_consensus.cc:2258] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:32:41.780196 24685 ts_tablet_manager.cc:1905] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:32:41.780592 24684 ts_tablet_manager.cc:1905] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:32:41.782624 24685 ts_tablet_manager.cc:1918] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:32:41.782745 24685 log.cc:1196] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/fbbb01542bdb4a67b0fcb55b866f9f1a
I20260809 16:32:41.782788 24684 ts_tablet_manager.cc:1918] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:32:41.782886 24684 log.cc:1196] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/fbbb01542bdb4a67b0fcb55b866f9f1a
I20260809 16:32:41.783103 24685 ts_tablet_manager.cc:1939] T fbbb01542bdb4a67b0fcb55b866f9f1a P f07e6d7cde2d40c2bcedd2c2b4cc5694: Deleting consensus metadata
I20260809 16:32:41.783219 24684 ts_tablet_manager.cc:1939] T fbbb01542bdb4a67b0fcb55b866f9f1a P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Deleting consensus metadata
I20260809 16:32:41.783974 23848 catalog_manager.cc:4841] TS f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689): tablet fbbb01542bdb4a67b0fcb55b866f9f1a (table test-table [id=e0c2d08b38cc4eb6aab8f95d762b17d0]) successfully deleted
I20260809 16:32:41.784394 23846 catalog_manager.cc:4841] TS a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333): tablet fbbb01542bdb4a67b0fcb55b866f9f1a (table test-table [id=e0c2d08b38cc4eb6aab8f95d762b17d0]) successfully deleted
W20260809 16:32:41.786696 23848 connection.cc:535] client connection to 127.30.23.129:45881 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 16:32:41.786836 23848 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:32:41.786924 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 24202
I20260809 16:32:41.801991 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:32:41.798476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45881 (local address 127.0.0.1:57338)
0809 16:32:41.801839 (+ 3363us) negotiation.cc:107] Waiting for socket to connect
0809 16:32:41.801935 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:45881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3321}
W20260809 16:32:41.802448 23848 catalog_manager.cc:4584] TS 98cdb8a0a78c4179b6421b96d1658f9d (127.30.23.129:45881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fbbb01542bdb4a67b0fcb55b866f9f1a: Network error: Client connection negotiation failed: client connection to 127.30.23.129:45881: connect: Connection refused (error 111)
I20260809 16:32:41.812513 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 24418
I20260809 16:32:41.832556 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 23818
I20260809 16:33:01.852831 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:33381
--webserver_interface=127.30.23.190
--webserver_port=46495
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:33381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:01.936205 24688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:01.936470 24688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:01.936509 24688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:01.937681 24688 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 16:33:01.937747 24688 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 16:33:01.937772 24688 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 16:33:01.937791 24688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:01.937820 24688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:01.937850 24688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:01.937873 24688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:01.937901 24688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:01.939406 24688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:33381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:33381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=46495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:01.939693 24688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:01.939930 24688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:01.949878 24700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:01.949878 24701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:01.950069 24688 server_base.cc:1034] running on GCE node
W20260809 16:33:01.950296 24704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:01.950661 24688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:01.951073 24688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:01.952253 24688 hybrid_clock.cc:648] HybridClock initialized: now 1786293181952252 us; error 44 us; skew 500 ppm
Aug 09 16:33:01 dist-test-slave-j2vl krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:01.958189 24688 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:33:01.958726 24688 webserver.cc:460] Webserver started at http://127.30.23.190:46495/ using document root <none> and password file <none>
I20260809 16:33:01.958990 24688 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:01.959074 24688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:01.960630 24688 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:33:01.961526 24715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:01.961787 24688 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:33:01.961925 24688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "d0bfbed6462a44a4aee381609e12ca24"
format_stamp: "Formatted at 2026-08-09 16:32:40 on dist-test-slave-j2vl"
server_key: "234518f36f970b678d4632a0052aa159"
server_key_iv: "ac5b8f3ec09bcfa1d33d8faaa0e70c3a"
server_key_version: "encryptionkey@0"
I20260809 16:33:01.962388 24688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:01.994333 24688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:01.996551 24688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:01.996731 24688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:02.001660 24688 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:33381
I20260809 16:33:02.001708 24807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:33381 every 8 connection(s)
I20260809 16:33:02.002154 24688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:02.003698 24808 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:33:02.004328 24808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Bootstrap starting.
I20260809 16:33:02.009720 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 24688
I20260809 16:33:02.010076 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:45881
--local_ip_for_outbound_sockets=127.30.23.129
--tserver_master_addrs=127.30.23.190:33381
--webserver_port=36257
--webserver_interface=127.30.23.129
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:33:02.013572 24808 log.cc:824] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:02.015568 24808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:33:02.015882 24808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Bootstrap complete.
I20260809 16:33:02.018286 24808 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } }
I20260809 16:33:02.018517 24808 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0bfbed6462a44a4aee381609e12ca24, State: Initialized, Role: FOLLOWER
I20260809 16:33:02.018677 24808 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } }
I20260809 16:33:02.018759 24808 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:02.018821 24808 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:02.018887 24808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:33:02.019903 24808 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } }
I20260809 16:33:02.020020 24808 leader_election.cc:302] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0bfbed6462a44a4aee381609e12ca24; no voters:
I20260809 16:33:02.020206 24808 leader_election.cc:288] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:33:02.020242 24815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:33:02.020365 24815 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 2 LEADER]: Becoming Leader. State: Replica: d0bfbed6462a44a4aee381609e12ca24, State: Running, Role: LEADER
I20260809 16:33:02.020501 24808 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:02.020521 24815 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } }
I20260809 16:33:02.020906 24816 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d0bfbed6462a44a4aee381609e12ca24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } } }
I20260809 16:33:02.021023 24816 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:02.021356 24817 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0bfbed6462a44a4aee381609e12ca24. Latest consensus state: current_term: 2 leader_uuid: "d0bfbed6462a44a4aee381609e12ca24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } } }
I20260809 16:33:02.021456 24817 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:02.021519 24827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:02.022336 24827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:02.022490 24827 catalog_manager.cc:1259] Loaded cluster ID: b419b61700ab41e1b5f160c1a503bfd1
I20260809 16:33:02.022792 24827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:33:02.024196 24827 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:02.024457 24827 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Loaded TSK: 0
I20260809 16:33:02.043340 24827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Generated new TSK 1
I20260809 16:33:02.043457 24827 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:33:02.104046 24813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:02.104278 24813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:02.104303 24813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:02.105607 24813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:02.105659 24813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:02.105686 24813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:33:02.107197 24813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:45881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=36257
--tserver_master_addrs=127.30.23.190:33381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:02.107573 24813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:02.107864 24813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:02.114807 24853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:02.114858 24858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:02.114851 24855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:02.115422 24813 server_base.cc:1034] running on GCE node
I20260809 16:33:02.115685 24813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:02.116032 24813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:02.117233 24813 hybrid_clock.cc:648] HybridClock initialized: now 1786293182117203 us; error 42 us; skew 500 ppm
Aug 09 16:33:02 dist-test-slave-j2vl krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293182, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:02.120671 24813 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:33:02.121258 24813 webserver.cc:460] Webserver started at http://127.30.23.129:36257/ using document root <none> and password file <none>
I20260809 16:33:02.121553 24813 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:02.121642 24813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:02.123245 24813 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:02.124118 24867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:02.124326 24813 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:02.124444 24813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "98cdb8a0a78c4179b6421b96d1658f9d"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "671e76e497dc3a28d81a3d936135bc87"
server_key_iv: "b25ffb7a91acaa1576aca80f2d52d50d"
server_key_version: "encryptionkey@0"
I20260809 16:33:02.124836 24813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:02.140283 24813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:02.142831 24813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:02.143002 24813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:02.143251 24813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:02.143711 24877 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:33:02.145861 24813 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:33:02.145951 24813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:33:02.145990 24813 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:33:02.146705 24813 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:33:02.146760 24813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:33:02.146822 24877 tablet_bootstrap.cc:492] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Bootstrap starting.
I20260809 16:33:02.154521 24813 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:45881
I20260809 16:33:02.154562 25040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:45881 every 8 connection(s)
I20260809 16:33:02.155004 24813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:33:02.156625 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 24813
I20260809 16:33:02.156975 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:36689
--local_ip_for_outbound_sockets=127.30.23.130
--tserver_master_addrs=127.30.23.190:33381
--webserver_port=35585
--webserver_interface=127.30.23.130
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:33:02.159339 24877 log.cc:824] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:02.162848 24877 tablet_bootstrap.cc:492] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:33:02.163236 24877 tablet_bootstrap.cc:492] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Bootstrap complete.
Aug 09 16:33:02 dist-test-slave-j2vl krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293182, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:33:02.164579 24877 ts_tablet_manager.cc:1397] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.007s
I20260809 16:33:02.166524 24877 raft_consensus.cc:348] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:02.167013 24877 raft_consensus.cc:729] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98cdb8a0a78c4179b6421b96d1658f9d, State: Initialized, Role: FOLLOWER
I20260809 16:33:02.167202 24877 consensus_queue.cc:260] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:02.167559 24877 ts_tablet_manager.cc:1428] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
I20260809 16:33:02.168803 25045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.155763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:33003)
0809 16:33:02.156060 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:02.156106 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:02.156866 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:02.157787 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:02.157791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:02.158205 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:02.158504 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.158512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.160343 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.160344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.161533 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.161537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.162476 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.162478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.162638 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.163222 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:02.163231 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:02.164797 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:02.166869 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:02.166877 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:02.166879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:02.167183 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:02.167522 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:02.167523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:02.167524 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:02.167676 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:02.168277 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:02.168284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:02.168411 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 16:33:02.168514 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 16:33:02.168655 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":103,"threads_started":1}
I20260809 16:33:02.169037 25046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.155992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:33003 (local address 127.30.23.190:33381)
0809 16:33:02.156211 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:02.156220 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:02.156256 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:33:02.157434 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:02.157441 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:02.157504 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:02.157637 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:02.158651 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.160190 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.161648 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.162227 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.162328 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.164908 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:02.164913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:02.164913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:02.164929 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:02.166778 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:02.167265 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:02.167268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:02.167269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:02.167432 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:02.167770 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:02.167772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:02.167772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:02.168156 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:02.168282 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:02.168610 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 16:33:02.168785 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":91,"threads_started":1}
I20260809 16:33:02.169765 25041 heartbeater.cc:347] Connected to a master server at 127.30.23.190:33381
I20260809 16:33:02.169890 25041 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:02.170104 25041 heartbeater.cc:511] Master 127.30.23.190:33381 requested a full tablet report, sending...
I20260809 16:33:02.170871 24753 ts_manager.cc:194] Registered new tserver with Master: 98cdb8a0a78c4179b6421b96d1658f9d (127.30.23.129:45881)
I20260809 16:33:02.172047 24753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:33003
I20260809 16:33:02.173050 25041 heartbeater.cc:503] Master 127.30.23.190:33381 was elected leader, sending a full tablet report...
Aug 09 16:33:02 dist-test-slave-j2vl krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293181, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for kudu/127.30.23.129@KRBTEST.COM
I20260809 16:33:02.180142 25055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.171502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45881 (local address 127.0.0.1:55174)
0809 16:33:02.171743 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:02.171773 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:02.171834 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:02.172665 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:02.172673 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:02.172952 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:02.173003 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.173005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.174051 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.174052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.174624 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.174626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.175119 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.175120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.175200 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.175713 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:02.175718 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:02.177067 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:02.178904 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:02.178913 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:02.178915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:02.179126 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:02.179363 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:02.179364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:02.179364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:02.179438 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:02.179730 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:02.179737 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:02.179792 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 16:33:02.179887 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 16:33:02.180036 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":117,"threads_started":1}
I20260809 16:33:02.180142 25058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.172350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55174 (local address 127.30.23.129:45881)
0809 16:33:02.172516 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:02.172520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:02.172531 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:33:02.172568 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:02.172570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:02.172581 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:02.172662 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:02.173086 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.173970 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.174718 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.175027 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.175116 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.177156 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:02.177165 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:02.177165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:02.177178 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:02.178803 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:02.179226 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:02.179229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:02.179230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:02.179291 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:02.179508 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:02.179509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:02.179509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:02.179661 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:02.179752 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:02.179949 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 16:33:02.180047 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":105,"threads_started":1}
I20260809 16:33:02.180562 24938 tablet_service.cc:1514] Processing DeleteTablet for tablet fbbb01542bdb4a67b0fcb55b866f9f1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:32:41 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:55174
I20260809 16:33:02.181084 25059 tablet_replica.cc:330] stopping tablet replica
I20260809 16:33:02.181205 25059 raft_consensus.cc:2229] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:02.181279 25059 raft_consensus.cc:2258] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:02.181636 25059 ts_tablet_manager.cc:1905] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:02.183817 25059 ts_tablet_manager.cc:1918] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:02.183882 25059 log.cc:1196] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal/wals/fbbb01542bdb4a67b0fcb55b866f9f1a
I20260809 16:33:02.184124 25059 ts_tablet_manager.cc:1939] T fbbb01542bdb4a67b0fcb55b866f9f1a P 98cdb8a0a78c4179b6421b96d1658f9d: Deleting consensus metadata
I20260809 16:33:02.184677 24723 catalog_manager.cc:4841] TS 98cdb8a0a78c4179b6421b96d1658f9d (127.30.23.129:45881): tablet fbbb01542bdb4a67b0fcb55b866f9f1a (table test-table [id=e0c2d08b38cc4eb6aab8f95d762b17d0]) successfully deleted
W20260809 16:33:02.248122 25047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:02.248365 25047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:02.248404 25047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:02.249614 25047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:02.249692 25047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:02.249742 25047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:33:02.251283 25047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:36689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=35585
--tserver_master_addrs=127.30.23.190:33381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:02.251561 25047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:02.251809 25047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:02.257520 25067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:02.257513 25065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:02.257513 25069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:02.258070 25047 server_base.cc:1034] running on GCE node
I20260809 16:33:02.258359 25047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:02.258654 25047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:02.259827 25047 hybrid_clock.cc:648] HybridClock initialized: now 1786293182259791 us; error 57 us; skew 500 ppm
Aug 09 16:33:02 dist-test-slave-j2vl krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293182, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:02.262643 25047 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:33:02.263087 25047 webserver.cc:460] Webserver started at http://127.30.23.130:35585/ using document root <none> and password file <none>
I20260809 16:33:02.263332 25047 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:02.263422 25047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:02.264878 25047 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:33:02.265527 25081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:02.265689 25047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:02.265805 25047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "be085fa6b245c786ca8aacbc1100e21d"
server_key_iv: "000b1cdcbc46fdd2ffc9299ff56ec997"
server_key_version: "encryptionkey@0"
I20260809 16:33:02.266208 25047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:02.273898 25047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:02.275967 25047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:02.276105 25047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:02.276343 25047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:02.276650 25047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:02.276683 25047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:02.276723 25047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:02.276750 25047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:02.282829 25047 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:36689
I20260809 16:33:02.282876 25273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:36689 every 8 connection(s)
I20260809 16:33:02.283221 25047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:33:02.285203 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 25047
I20260809 16:33:02.285537 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:41333
--local_ip_for_outbound_sockets=127.30.23.131
--tserver_master_addrs=127.30.23.190:33381
--webserver_port=41579
--webserver_interface=127.30.23.131
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:33:02 dist-test-slave-j2vl krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293182, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:33:02.293102 25046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.283950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:47903 (local address 127.30.23.190:33381)
0809 16:33:02.284000 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:02.284002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:02.284273 (+ 271us) server_negotiation.cc:407] Connection header received
0809 16:33:02.284818 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:02.284820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:02.284834 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:02.284887 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:02.286123 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.286640 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.287664 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.287877 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.287967 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.290285 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:02.290287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:02.290287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:02.290296 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:02.291705 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:02.292111 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:02.292113 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:02.292113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:02.292176 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:02.292428 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:02.292429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:02.292430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:02.292551 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:02.292628 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:02.292908 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 16:33:02.293006 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:33:02.293287 25278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.283952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:47903)
0809 16:33:02.284119 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:02.284179 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:02.284704 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:02.284998 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:02.285002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:02.285793 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:02.286005 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.286010 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.286752 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.286753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.287568 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.287570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.287989 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.287990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.288128 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.288686 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:02.288695 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:02.290168 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:02.291789 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:02.291797 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:02.291800 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:02.292035 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:02.292251 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:02.292252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:02.292252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:02.292360 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:02.292618 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:02.292625 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:02.292768 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 16:33:02.292879 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 16:33:02.293042 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260809 16:33:02.293915 25275 heartbeater.cc:347] Connected to a master server at 127.30.23.190:33381
I20260809 16:33:02.294018 25275 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:02.294224 25275 heartbeater.cc:511] Master 127.30.23.190:33381 requested a full tablet report, sending...
I20260809 16:33:02.294659 24753 ts_manager.cc:194] Registered new tserver with Master: f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689)
I20260809 16:33:02.295245 24753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:47903
W20260809 16:33:02.370746 25280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:02.370961 25280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:02.370986 25280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:02.372085 25280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:02.372124 25280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:02.372150 25280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:33:02.373631 25280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:41333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=41579
--tserver_master_addrs=127.30.23.190:33381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:02.373828 25280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:02.374019 25280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:02.379235 25293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:02.379243 25298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:02.379283 25292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:02.379496 25280 server_base.cc:1034] running on GCE node
I20260809 16:33:02.379739 25280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:02.380033 25280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:02.381194 25280 hybrid_clock.cc:648] HybridClock initialized: now 1786293182381181 us; error 25 us; skew 500 ppm
Aug 09 16:33:02 dist-test-slave-j2vl krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293182, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:02.384006 25280 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:33:02.384496 25280 webserver.cc:460] Webserver started at http://127.30.23.131:41579/ using document root <none> and password file <none>
I20260809 16:33:02.384703 25280 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:02.384773 25280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:02.386281 25280 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:33:02.386838 25308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:02.387038 25280 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:33:02.387123 25280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "dc78da247c0438e66c96898b66d64ea3"
server_key_iv: "7e154dd1c185176431307ba7dc321982"
server_key_version: "encryptionkey@0"
I20260809 16:33:02.387459 25280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:02.415923 25280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:02.418251 25280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:02.418392 25280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:02.418601 25280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:02.418941 25280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:02.418977 25280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:02.419001 25280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:02.419041 25280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:02.425460 25280 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:41333
I20260809 16:33:02.425501 25484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:41333 every 8 connection(s)
I20260809 16:33:02.425858 25280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:33:02.431861 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 25280
Aug 09 16:33:02 dist-test-slave-j2vl krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293182, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:33:02.435675 25046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.426668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:37289 (local address 127.30.23.190:33381)
0809 16:33:02.426756 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:02.426758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:02.426932 (+ 174us) server_negotiation.cc:407] Connection header received
0809 16:33:02.427487 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:02.427489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:02.427499 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:02.427566 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:02.428597 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.429097 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.430125 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.430314 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.430373 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.432810 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:02.432811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:02.432811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:02.432817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:02.434304 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:02.434697 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:02.434699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:02.434709 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:02.434772 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:02.435034 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:02.435035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:02.435035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:02.435147 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:02.435222 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:02.435503 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 16:33:02.435586 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 16:33:02.435884 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.426636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:37289)
0809 16:33:02.426795 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:02.426839 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:02.427396 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:02.427619 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:02.427622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:02.428264 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:02.428502 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.428507 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.429186 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.429187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.430041 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.430044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.430386 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.430387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.430486 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.430971 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:02.430979 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:02.432723 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:02.434387 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:02.434394 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:02.434396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:02.434620 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:02.434848 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:02.434849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:02.434849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:02.434960 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:02.435212 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:02.435219 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:02.435363 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 16:33:02.435468 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 16:33:02.435628 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260809 16:33:02.436389 25487 heartbeater.cc:347] Connected to a master server at 127.30.23.190:33381
I20260809 16:33:02.436519 25487 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:02.436756 25487 heartbeater.cc:511] Master 127.30.23.190:33381 requested a full tablet report, sending...
I20260809 16:33:02.437252 24753 ts_manager.cc:194] Registered new tserver with Master: a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333)
I20260809 16:33:02.437765 24753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:37289
I20260809 16:33:02.439715 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.432367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39078 (local address 127.30.23.190:33381)
0809 16:33:02.432497 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:02.432499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:02.432587 (+ 88us) server_negotiation.cc:407] Connection header received
0809 16:33:02.432650 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:02.432651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:02.432661 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:02.432727 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:02.433317 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.434023 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.434901 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.435117 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.435191 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.436772 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:02.436773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:02.436773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:02.436780 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:02.438389 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:02.438788 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:02.438789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:02.438789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:02.438849 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:02.439188 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:02.439189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:02.439189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:02.439315 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:02.439464 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:02.439572 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 16:33:02.439645 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260809 16:33:02.440572 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:33:02.443670 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.440964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39082 (local address 127.30.23.190:33381)
0809 16:33:02.441034 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:02.441036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:02.441175 (+ 139us) server_negotiation.cc:407] Connection header received
0809 16:33:02.441213 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:02.441214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:02.441246 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:02.441319 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:33:02.441518 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.441982 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.442794 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.442998 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.443050 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.443363 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:33:02.443421 (+ 58us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 16:33:02.443626 (+ 205us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39082: token expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 16:33:02.443730 25491 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39082: token expired
W20260809 16:33:02.443951 24635 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.23.190:33381)
I20260809 16:33:02.450528 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.444306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39098 (local address 127.30.23.190:33381)
0809 16:33:02.444383 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:02.444385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:02.444394 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:33:02.444411 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:02.444411 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:02.444420 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:02.444477 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:02.444985 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.445511 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.446329 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.446525 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.446591 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.447776 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:02.447777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:02.447777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:02.447785 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:02.449214 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:02.449636 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:02.449637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:02.449638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:02.449705 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:02.449992 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:02.449994 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:02.449994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:02.450114 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:02.450205 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:02.450349 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 16:33:02.450415 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 16:33:02.451833 24753 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:33:02.453553 24753 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:33:02.456255 24938 tablet_service.cc:1467] Processing CreateTablet for tablet 94d7c17b0fc94baab736e9faa455408e (DEFAULT_TABLE table=test-table [id=751c6052dc3e470597a61f4469d09bde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:02.456470 24938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94d7c17b0fc94baab736e9faa455408e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:02.457777 24877 tablet_bootstrap.cc:492] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d: Bootstrap starting.
I20260809 16:33:02.458087 24877 tablet_bootstrap.cc:654] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:02.459626 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.455638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:41333 (local address 127.0.0.1:58164)
0809 16:33:02.455861 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:02.455871 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:02.455915 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:02.456370 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:02.456371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:02.456374 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:02.456423 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.456424 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.457692 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.457693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.458596 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.458599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.459360 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.459361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.459426 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.459449 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 16:33:02.459510 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 16:33:02.459550 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":23,"thread_start_us":129,"threads_started":1}
I20260809 16:33:02.459693 24877 tablet_bootstrap.cc:492] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d: No bootstrap required, opened a new log
I20260809 16:33:02.459756 24877 ts_tablet_manager.cc:1397] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:33:02.459887 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.455951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58164 (local address 127.30.23.131:41333)
0809 16:33:02.456136 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:02.456140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:02.456150 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:33:02.456186 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:02.456188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:02.456203 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:02.456277 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:02.456504 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.457597 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.458752 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.459270 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.459359 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.459420 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:02.459545 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 16:33:02.459601 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260809 16:33:02.459913 24877 raft_consensus.cc:348] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:02.459988 24877 raft_consensus.cc:374] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:02.460043 24877 raft_consensus.cc:729] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98cdb8a0a78c4179b6421b96d1658f9d, State: Initialized, Role: FOLLOWER
I20260809 16:33:02.460089 24877 consensus_queue.cc:260] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:02.460189 24877 ts_tablet_manager.cc:1428] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:02.460608 25055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.455638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:36689 (local address 127.0.0.1:49368)
0809 16:33:02.455762 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:02.455774 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:02.455844 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:02.456261 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:02.456262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:02.456264 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:02.456320 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.456322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.458657 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.458658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.459427 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.459430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.460355 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.460356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.460426 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.460430 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:02.460489 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:33:02.460536 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260809 16:33:02.460588 25380 tablet_service.cc:1467] Processing CreateTablet for tablet 94d7c17b0fc94baab736e9faa455408e (DEFAULT_TABLE table=test-table [id=751c6052dc3e470597a61f4469d09bde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:02.460842 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.455876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49368 (local address 127.30.23.130:36689)
0809 16:33:02.456057 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:02.456062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:02.456077 (+ 15us) server_negotiation.cc:407] Connection header received
0809 16:33:02.456122 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:02.456124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:02.456143 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:02.456245 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:02.456388 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.457597 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.459493 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.460212 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.460334 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.460397 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:02.460705 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 16:33:02.460761 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260809 16:33:02.460958 25380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94d7c17b0fc94baab736e9faa455408e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:02.461572 25162 tablet_service.cc:1467] Processing CreateTablet for tablet 94d7c17b0fc94baab736e9faa455408e (DEFAULT_TABLE table=test-table [id=751c6052dc3e470597a61f4469d09bde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:02.461911 25162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94d7c17b0fc94baab736e9faa455408e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:02.464839 25499 tablet_bootstrap.cc:492] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Bootstrap starting.
I20260809 16:33:02.465669 25499 tablet_bootstrap.cc:654] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:02.465885 25501 tablet_bootstrap.cc:492] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694: Bootstrap starting.
I20260809 16:33:02.466281 25499 log.cc:824] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:02.466674 25501 tablet_bootstrap.cc:654] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:02.467197 25499 tablet_bootstrap.cc:492] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0: No bootstrap required, opened a new log
I20260809 16:33:02.467268 25499 ts_tablet_manager.cc:1397] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 16:33:02.467263 25501 log.cc:824] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:02.468107 25501 tablet_bootstrap.cc:492] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694: No bootstrap required, opened a new log
I20260809 16:33:02.468194 25501 ts_tablet_manager.cc:1397] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:33:02.469233 25499 raft_consensus.cc:348] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:02.469440 25499 raft_consensus.cc:374] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:02.469480 25499 raft_consensus.cc:729] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6267fa1a6464ae6a3b1c47f7fbcc4d0, State: Initialized, Role: FOLLOWER
I20260809 16:33:02.469619 25499 consensus_queue.cc:260] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:02.469911 25499 ts_tablet_manager.cc:1428] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 16:33:02.470032 25487 heartbeater.cc:503] Master 127.30.23.190:33381 was elected leader, sending a full tablet report...
I20260809 16:33:02.470294 25501 raft_consensus.cc:348] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:02.470468 25501 raft_consensus.cc:374] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:02.470528 25501 raft_consensus.cc:729] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f07e6d7cde2d40c2bcedd2c2b4cc5694, State: Initialized, Role: FOLLOWER
I20260809 16:33:02.470657 25501 consensus_queue.cc:260] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:02.470965 25501 ts_tablet_manager.cc:1428] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:33:02.471021 25275 heartbeater.cc:503] Master 127.30.23.190:33381 was elected leader, sending a full tablet report...
I20260809 16:33:02.488617 25051 raft_consensus.cc:482] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:33:02.488807 25051 raft_consensus.cc:504] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:02.489210 25051 leader_election.cc:288] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689), a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333)
I20260809 16:33:02.492216 25045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.489464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:41333 (local address 127.30.23.129:42473)
0809 16:33:02.489524 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:02.489536 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:02.489614 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:02.489805 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:02.489806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:02.489808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:02.489880 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.489881 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.490514 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.490515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.491369 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.491371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.491989 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.491990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.492048 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.492068 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 16:33:02.492115 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:33:02.492147 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 16:33:02.492246 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.489597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:42473 (local address 127.30.23.131:41333)
0809 16:33:02.489644 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:02.489646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:02.489666 (+ 20us) server_negotiation.cc:407] Connection header received
0809 16:33:02.489696 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:02.489697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:02.489708 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:02.489779 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:02.489968 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.490436 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.491460 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.491914 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.491986 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.491994 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:02.492136 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 16:33:02.492173 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:33:02.492590 25420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94d7c17b0fc94baab736e9faa455408e" candidate_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" is_pre_election: true
I20260809 16:33:02.492674 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.489478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:54835 (local address 127.30.23.130:36689)
0809 16:33:02.489537 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:02.489539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:02.489840 (+ 301us) server_negotiation.cc:407] Connection header received
0809 16:33:02.489873 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:02.489878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:02.489888 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:02.489953 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:02.490133 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.490632 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.491721 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.492269 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.492342 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.492352 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:02.492556 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 16:33:02.492601 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:33:02.492753 25420 raft_consensus.cc:2454] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98cdb8a0a78c4179b6421b96d1658f9d in term 0.
I20260809 16:33:02.492955 25507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.489525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:36689 (local address 127.30.23.129:54835)
0809 16:33:02.489708 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:02.489720 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:02.489794 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:02.489957 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:02.489958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:02.489959 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:02.490060 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.490061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.490766 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.490767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.491637 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:02.491640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.492388 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.492389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:02.492467 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.492472 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:33:02.492794 (+ 322us) client_negotiation.cc:240] Negotiation successful
0809 16:33:02.492834 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":129,"threads_started":1}
I20260809 16:33:02.493029 24872 leader_election.cc:302] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98cdb8a0a78c4179b6421b96d1658f9d, a6267fa1a6464ae6a3b1c47f7fbcc4d0; no voters:
I20260809 16:33:02.493165 25051 raft_consensus.cc:2781] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:33:02.493233 25051 raft_consensus.cc:482] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:33:02.493281 25051 raft_consensus.cc:3037] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:02.493302 25192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94d7c17b0fc94baab736e9faa455408e" candidate_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" is_pre_election: true
I20260809 16:33:02.493487 25192 raft_consensus.cc:2454] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98cdb8a0a78c4179b6421b96d1658f9d in term 0.
I20260809 16:33:02.494096 25051 raft_consensus.cc:504] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:02.494251 25051 leader_election.cc:288] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 1 election: Requested vote from peers f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689), a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333)
I20260809 16:33:02.494426 25420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94d7c17b0fc94baab736e9faa455408e" candidate_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0"
I20260809 16:33:02.494503 25420 raft_consensus.cc:3037] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:02.495231 25420 raft_consensus.cc:2454] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98cdb8a0a78c4179b6421b96d1658f9d in term 1.
I20260809 16:33:02.495390 25192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94d7c17b0fc94baab736e9faa455408e" candidate_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694"
I20260809 16:33:02.495424 24872 leader_election.cc:302] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98cdb8a0a78c4179b6421b96d1658f9d, a6267fa1a6464ae6a3b1c47f7fbcc4d0; no voters:
I20260809 16:33:02.495471 25192 raft_consensus.cc:3037] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:02.495541 25051 raft_consensus.cc:2781] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:02.495711 25051 raft_consensus.cc:686] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 LEADER]: Becoming Leader. State: Replica: 98cdb8a0a78c4179b6421b96d1658f9d, State: Running, Role: LEADER
I20260809 16:33:02.495795 25051 consensus_queue.cc:237] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:02.496467 25192 raft_consensus.cc:2454] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98cdb8a0a78c4179b6421b96d1658f9d in term 1.
I20260809 16:33:02.497033 24753 catalog_manager.cc:5495] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 98cdb8a0a78c4179b6421b96d1658f9d (127.30.23.129). New cstate: current_term: 1 leader_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } health_report { overall_health: HEALTHY } } }
I20260809 16:33:02.504990 25058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.502318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55180 (local address 127.30.23.129:45881)
0809 16:33:02.502376 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:02.502378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:02.502462 (+ 84us) server_negotiation.cc:407] Connection header received
0809 16:33:02.502519 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:02.502520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:02.502538 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:02.502611 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:33:02.502697 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.503217 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.504230 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:02.504448 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:02.504521 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:02.504650 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:33:02.504759 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:33:02.504840 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:02.504910 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 16:33:02.504933 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:33:02.506578 25192 raft_consensus.cc:1261] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 1 FOLLOWER]: Refusing update from remote peer 98cdb8a0a78c4179b6421b96d1658f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:02.506588 25420 raft_consensus.cc:1261] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 1 FOLLOWER]: Refusing update from remote peer 98cdb8a0a78c4179b6421b96d1658f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:02.506935 25051 consensus_queue.cc:1035] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:02.507093 25509 consensus_queue.cc:1035] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:02.510681 25516 mvcc.cc:205] Tried to move back new op lower bound from 7316656875544469504 to 7316656875503435776. Current Snapshot: MvccSnapshot[applied={T|T < 7316656875544469504}]
I20260809 16:33:02.512887 24752 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:02.512993 24752 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:02.513832 24752 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Sending DeleteTablet for 3 replicas of tablet 94d7c17b0fc94baab736e9faa455408e
I20260809 16:33:02.514158 25380 tablet_service.cc:1514] Processing DeleteTablet for tablet 94d7c17b0fc94baab736e9faa455408e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:02 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:58164
I20260809 16:33:02.514194 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 24813
I20260809 16:33:02.514286 25162 tablet_service.cc:1514] Processing DeleteTablet for tablet 94d7c17b0fc94baab736e9faa455408e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:02 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:49368
I20260809 16:33:02.533751 25534 tablet_replica.cc:330] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0: stopping tablet replica
I20260809 16:33:02.534000 25534 raft_consensus.cc:2229] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:02.534195 25534 raft_consensus.cc:2258] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:02.535310 25535 tablet_replica.cc:330] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694: stopping tablet replica
I20260809 16:33:02.535436 25535 raft_consensus.cc:2229] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:02.535553 25535 raft_consensus.cc:2258] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:02.535557 25534 ts_tablet_manager.cc:1905] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:02.536031 25535 ts_tablet_manager.cc:1905] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:02.538048 25534 ts_tablet_manager.cc:1918] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:02.538046 25535 ts_tablet_manager.cc:1918] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:02.538133 25534 log.cc:1196] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/94d7c17b0fc94baab736e9faa455408e
I20260809 16:33:02.538133 25535 log.cc:1196] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/94d7c17b0fc94baab736e9faa455408e
I20260809 16:33:02.538463 25535 ts_tablet_manager.cc:1939] T 94d7c17b0fc94baab736e9faa455408e P f07e6d7cde2d40c2bcedd2c2b4cc5694: Deleting consensus metadata
I20260809 16:33:02.538463 25534 ts_tablet_manager.cc:1939] T 94d7c17b0fc94baab736e9faa455408e P a6267fa1a6464ae6a3b1c47f7fbcc4d0: Deleting consensus metadata
I20260809 16:33:02.538956 24723 catalog_manager.cc:4841] TS f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689): tablet 94d7c17b0fc94baab736e9faa455408e (table test-table [id=751c6052dc3e470597a61f4469d09bde]) successfully deleted
I20260809 16:33:02.539078 24719 catalog_manager.cc:4841] TS a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333): tablet 94d7c17b0fc94baab736e9faa455408e (table test-table [id=751c6052dc3e470597a61f4469d09bde]) successfully deleted
W20260809 16:33:02.540231 24723 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:45881 (error 108)
I20260809 16:33:02.540814 25055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:02.540685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:45881 (local address 127.0.0.1:55190)
0809 16:33:02.540744 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:02.540783 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:45881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 16:33:02.540939 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 25047
W20260809 16:33:02.540954 24723 catalog_manager.cc:4584] TS 98cdb8a0a78c4179b6421b96d1658f9d (127.30.23.129:45881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94d7c17b0fc94baab736e9faa455408e: Network error: Client connection negotiation failed: client connection to 127.30.23.129:45881: connect: Connection refused (error 111)
I20260809 16:33:02.560002 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 25280
I20260809 16:33:02.569940 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 24688
I20260809 16:33:22.579877 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:33381
--webserver_interface=127.30.23.190
--webserver_port=46495
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:33381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:22.668329 25541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:22.668558 25541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:22.668582 25541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:22.669795 25541 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 16:33:22.669838 25541 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 16:33:22.669852 25541 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 16:33:22.669864 25541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:22.669878 25541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:22.669898 25541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:22.669909 25541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:22.669922 25541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:22.671537 25541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:33381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:33381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=46495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:22.671752 25541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:22.671942 25541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:22.677652 25552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:22.677671 25557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:22.677640 25551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:22.677691 25541 server_base.cc:1034] running on GCE node
I20260809 16:33:22.678277 25541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:22.678694 25541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:22.679898 25541 hybrid_clock.cc:648] HybridClock initialized: now 1786293202679883 us; error 29 us; skew 500 ppm
Aug 09 16:33:22 dist-test-slave-j2vl krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293202, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:22.687600 25541 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:33:22.688103 25541 webserver.cc:460] Webserver started at http://127.30.23.190:46495/ using document root <none> and password file <none>
I20260809 16:33:22.688478 25541 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:22.688553 25541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:22.690114 25541 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:33:22.690939 25571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:22.691175 25541 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:22.691293 25541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "d0bfbed6462a44a4aee381609e12ca24"
format_stamp: "Formatted at 2026-08-09 16:32:40 on dist-test-slave-j2vl"
server_key: "234518f36f970b678d4632a0052aa159"
server_key_iv: "ac5b8f3ec09bcfa1d33d8faaa0e70c3a"
server_key_version: "encryptionkey@0"
I20260809 16:33:22.691697 25541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:22.711326 25541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:22.713833 25541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:22.714015 25541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:22.718551 25541 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:33381
I20260809 16:33:22.718587 25668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:33381 every 8 connection(s)
I20260809 16:33:22.718978 25541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:22.720474 25670 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:33:22.721199 25670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Bootstrap starting.
I20260809 16:33:22.726877 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 25541
I20260809 16:33:22.727221 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:45881
--local_ip_for_outbound_sockets=127.30.23.129
--tserver_master_addrs=127.30.23.190:33381
--webserver_port=36257
--webserver_interface=127.30.23.129
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:33:22.730365 25670 log.cc:824] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:22.733965 25670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:33:22.734369 25670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Bootstrap complete.
I20260809 16:33:22.738673 25670 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } }
I20260809 16:33:22.739104 25670 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0bfbed6462a44a4aee381609e12ca24, State: Initialized, Role: FOLLOWER
I20260809 16:33:22.739302 25670 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } }
I20260809 16:33:22.739401 25670 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:22.739463 25670 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:22.739523 25670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 2 FOLLOWER]: Advancing to term 3
I20260809 16:33:22.741079 25670 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } }
I20260809 16:33:22.741200 25670 leader_election.cc:302] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0bfbed6462a44a4aee381609e12ca24; no voters:
I20260809 16:33:22.741418 25670 leader_election.cc:288] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 16:33:22.741508 25675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 16:33:22.741691 25670 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:22.741703 25675 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [term 3 LEADER]: Becoming Leader. State: Replica: d0bfbed6462a44a4aee381609e12ca24, State: Running, Role: LEADER
I20260809 16:33:22.741958 25675 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } }
I20260809 16:33:22.742319 25676 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d0bfbed6462a44a4aee381609e12ca24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } } }
I20260809 16:33:22.742319 25675 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0bfbed6462a44a4aee381609e12ca24. Latest consensus state: current_term: 3 leader_uuid: "d0bfbed6462a44a4aee381609e12ca24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bfbed6462a44a4aee381609e12ca24" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33381 } } }
I20260809 16:33:22.742556 25675 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:22.742547 25676 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:22.743031 25688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:22.744211 25688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:22.744577 25688 catalog_manager.cc:1259] Loaded cluster ID: b419b61700ab41e1b5f160c1a503bfd1
I20260809 16:33:22.744621 25688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:33:22.745543 25688 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:22.745741 25688 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Loaded TSK: 0
I20260809 16:33:22.745805 25688 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0bfbed6462a44a4aee381609e12ca24: Loaded TSK: 1
F20260809 16:33:22.746026 25688 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fca238d09ad google::LogMessage::Fail() at ??:0
@ 0x7fca238d388a google::LogMessage::SendToLog() at ??:0
@ 0x7fca238d04de google::LogMessage::Flush() at ??:0
@ 0x7fca238d1a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fca2603d104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fca2603ad41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fca2b82a1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fca2b82a48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fca2b82a594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fca2b7f0c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fca2b7faad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fca2579e0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fca25793954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fca265f76db start_thread at ??:0
@ 0x7fca223ce61f clone at ??:0
W20260809 16:33:22.850597 25672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:22.850900 25672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:22.850941 25672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:22.852600 25672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:22.852665 25672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:22.852707 25672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:33:22.854444 25672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:45881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=36257
--tserver_master_addrs=127.30.23.190:33381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:22.854753 25672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:22.855006 25672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:22.860461 25728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:22.860711 25672 server_base.cc:1034] running on GCE node
W20260809 16:33:22.860780 25729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:22.860464 25731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:22.861083 25672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:22.861409 25672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:22.862603 25672 hybrid_clock.cc:648] HybridClock initialized: now 1786293202862577 us; error 41 us; skew 500 ppm
Aug 09 16:33:22 dist-test-slave-j2vl krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293202, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:22.865896 25672 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:33:22.866482 25672 webserver.cc:460] Webserver started at http://127.30.23.129:36257/ using document root <none> and password file <none>
I20260809 16:33:22.866761 25672 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:22.866849 25672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:22.868726 25672 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:33:22.869920 25745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:22.870147 25672 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:22.870283 25672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "98cdb8a0a78c4179b6421b96d1658f9d"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "671e76e497dc3a28d81a3d936135bc87"
server_key_iv: "b25ffb7a91acaa1576aca80f2d52d50d"
server_key_version: "encryptionkey@0"
I20260809 16:33:22.870636 25672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:22.892087 25672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:22.894855 25672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:22.895015 25672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:22.895303 25672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:22.895882 25755 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:33:22.897953 25672 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:33:22.898042 25672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:33:22.898092 25672 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:33:22.899271 25672 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:33:22.899323 25672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:22.899505 25755 tablet_bootstrap.cc:492] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d: Bootstrap starting.
I20260809 16:33:22.907405 25672 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:45881
I20260809 16:33:22.907604 25921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:45881 every 8 connection(s)
I20260809 16:33:22.907912 25672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:33:22.912041 25755 log.cc:824] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:22.914880 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 25672
I20260809 16:33:22.915210 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:36689
--local_ip_for_outbound_sockets=127.30.23.130
--tserver_master_addrs=127.30.23.190:33381
--webserver_port=35585
--webserver_interface=127.30.23.130
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:33:22.915462 25755 tablet_bootstrap.cc:492] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:33:22.915848 25755 tablet_bootstrap.cc:492] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d: Bootstrap complete.
I20260809 16:33:22.917327 25755 ts_tablet_manager.cc:1397] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.004s
I20260809 16:33:22.919035 25755 raft_consensus.cc:348] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:22.919430 25755 raft_consensus.cc:729] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98cdb8a0a78c4179b6421b96d1658f9d, State: Initialized, Role: FOLLOWER
I20260809 16:33:22.919632 25755 consensus_queue.cc:260] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:22.920002 25755 ts_tablet_manager.cc:1428] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
W20260809 16:33:22.987332 25928 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:33:22.908770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:42087)
0809 16:33:22.909029 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:22.909076 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:22.909792 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:22.987102 (+ 77310us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":125,"threads_started":1}
W20260809 16:33:22.987630 25923 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:33381 (0 consecutive failures): Network error: Failed to ping master at 127.30.23.190:33381: Client connection negotiation failed: client connection to 127.30.23.190:33381: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:33:22.988024 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:22.987899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:42985)
0809 16:33:22.987956 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:22.987996 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 16:33:22.988438 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:22.988349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:43657)
0809 16:33:22.988398 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:22.988423 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 16:33:22.988598 25923 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:33:23.006647 25931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:23.006863 25931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:23.006887 25931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:23.008066 25931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:23.008109 25931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:23.008136 25931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:33:23.009622 25931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:36689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=35585
--tserver_master_addrs=127.30.23.190:33381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:23.009825 25931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:23.009999 25931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:23.015170 25939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:23.015177 25942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:23.015157 25938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:23.015372 25931 server_base.cc:1034] running on GCE node
I20260809 16:33:23.015627 25931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:23.015887 25931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:23.017071 25931 hybrid_clock.cc:648] HybridClock initialized: now 1786293203017067 us; error 47 us; skew 500 ppm
Aug 09 16:33:23 dist-test-slave-j2vl krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293203, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:23.019827 25931 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:33:23.020238 25931 webserver.cc:460] Webserver started at http://127.30.23.130:35585/ using document root <none> and password file <none>
I20260809 16:33:23.020468 25931 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:23.020538 25931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:23.021984 25931 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:33:23.022691 25949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:23.022946 25931 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:33:23.023066 25931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "be085fa6b245c786ca8aacbc1100e21d"
server_key_iv: "000b1cdcbc46fdd2ffc9299ff56ec997"
server_key_version: "encryptionkey@0"
I20260809 16:33:23.023412 25931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:23.049734 25931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:23.052232 25931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:23.052413 25931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:23.052678 25931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:23.053020 25931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:23.053076 25931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:23.053113 25931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:23.053160 25931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:23.060083 25931 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:36689
I20260809 16:33:23.060134 26158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:36689 every 8 connection(s)
I20260809 16:33:23.060485 25931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:33:23.061682 26162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:23.061189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:50743)
0809 16:33:23.061456 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:23.061539 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":90,"threads_started":1}
W20260809 16:33:23.062100 26159 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:33381 (0 consecutive failures): Network error: Failed to ping master at 127.30.23.190:33381: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
I20260809 16:33:23.062394 26162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:23.062302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:39577)
0809 16:33:23.062349 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:23.062376 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:33:23.062708 26162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:23.062584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:54099)
0809 16:33:23.062654 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:23.062686 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 16:33:23.062894 26159 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:33:23.064311 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 25931
I20260809 16:33:23.064648 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:41333
--local_ip_for_outbound_sockets=127.30.23.131
--tserver_master_addrs=127.30.23.190:33381
--webserver_port=41579
--webserver_interface=127.30.23.131
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:23.154618 26164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:23.154873 26164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:23.154901 26164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:23.156070 26164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:23.156112 26164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:23.156138 26164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:33:23.157716 26164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:41333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=41579
--tserver_master_addrs=127.30.23.190:33381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:23.157960 26164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:23.158236 26164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:23.163870 26172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:23.164013 26164 server_base.cc:1034] running on GCE node
W20260809 16:33:23.163887 26173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:23.163902 26178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:23.164518 26164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:23.164774 26164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:23.165967 26164 hybrid_clock.cc:648] HybridClock initialized: now 1786293203165956 us; error 33 us; skew 500 ppm
Aug 09 16:33:23 dist-test-slave-j2vl krb5kdc[23797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293203, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:23.168661 26164 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:33:23.169174 26164 webserver.cc:460] Webserver started at http://127.30.23.131:41579/ using document root <none> and password file <none>
I20260809 16:33:23.169464 26164 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:23.169517 26164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:23.170950 26164 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:33:23.171691 26189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:23.171875 26164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:33:23.171988 26164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0"
format_stamp: "Formatted at 2026-08-09 16:32:41 on dist-test-slave-j2vl"
server_key: "dc78da247c0438e66c96898b66d64ea3"
server_key_iv: "7e154dd1c185176431307ba7dc321982"
server_key_version: "encryptionkey@0"
I20260809 16:33:23.172353 26164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:23.192165 26164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:23.194952 26164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:23.195147 26164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:23.195415 26164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:23.195816 26164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:23.195926 26164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:23.195978 26164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:23.196004 26164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:23.202280 26164 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:41333
I20260809 16:33:23.202443 26380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:41333 every 8 connection(s)
I20260809 16:33:23.202754 26164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:33:23.203920 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:23.203514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:43771)
0809 16:33:23.203690 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:23.203766 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":69,"threads_started":1}
W20260809 16:33:23.204607 26381 heartbeater.cc:644] Failed to heartbeat to 127.30.23.190:33381 (0 consecutive failures): Network error: Failed to ping master at 127.30.23.190:33381: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
I20260809 16:33:23.204965 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:23.204856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:60831)
0809 16:33:23.204909 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:23.204945 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 16:33:23.205283 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:23.205159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:53087)
0809 16:33:23.205231 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:23.205262 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 16:33:23.205478 26381 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:33:23.211414 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 26164
I20260809 16:33:23.989328 26386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:23.989054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:40057)
0809 16:33:23.989206 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:23.989245 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260809 16:33:24.063609 26387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:24.063338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:59787)
0809 16:33:24.063507 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:24.063552 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
I20260809 16:33:24.206101 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:24.205850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:58231)
0809 16:33:24.206008 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:24.206045 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260809 16:33:24.225925 26390 raft_consensus.cc:482] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:33:24.226058 26390 raft_consensus.cc:504] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:24.226444 26390 leader_election.cc:288] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689), a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333)
Aug 09 16:33:24 dist-test-slave-j2vl krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293202, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.130@KRBTEST.COM
Aug 09 16:33:24 dist-test-slave-j2vl krb5kdc[23797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293202, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.131@KRBTEST.COM
I20260809 16:33:24.240123 26386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:24.226631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:41333 (local address 127.30.23.129:45179)
0809 16:33:24.226929 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:24.226940 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:24.226982 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:24.227850 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:24.227854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:24.228561 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:24.228852 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:24.228860 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:24.230663 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:24.230665 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:24.231995 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:24.231998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:24.232926 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:24.232927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:24.233099 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:24.233861 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:24.233887 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:24.236429 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:24.238444 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:24.238445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:24.238446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:24.238656 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:24.238933 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:24.238933 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:24.238934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:24.239022 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:24.239566 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:24.239568 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:24.239695 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 16:33:24.239811 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 16:33:24.240025 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":92}
I20260809 16:33:24.240137 26393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:24.226763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:45179 (local address 127.30.23.131:41333)
0809 16:33:24.226957 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:24.226964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:24.226994 (+ 30us) server_negotiation.cc:407] Connection header received
0809 16:33:24.227539 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:24.227542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:24.227600 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:24.227757 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:24.229029 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:24.230477 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:24.232119 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:24.232805 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:24.232914 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:24.236558 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:24.236567 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:24.236567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:24.236578 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:24.238321 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:24.238743 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:24.238746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:24.238747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:24.238851 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:24.239111 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:24.239113 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:24.239113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:24.239466 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:24.239586 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:24.239883 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 16:33:24.240012 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260809 16:33:24.240172 26392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:24.226752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:54265 (local address 127.30.23.130:36689)
0809 16:33:24.226874 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:24.226887 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:24.226926 (+ 39us) server_negotiation.cc:407] Connection header received
0809 16:33:24.227539 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:24.227542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:24.227600 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:24.227757 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:24.229029 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:24.230476 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:24.232118 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:24.232782 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:24.232914 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:24.236193 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:24.236198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:24.236198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:24.236213 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:24.238202 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:24.238711 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:24.238714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:24.238715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:24.238816 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:24.239095 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:24.239096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:24.239096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:24.239353 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:24.239567 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:24.239870 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 16:33:24.240050 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":55,"threads_started":1}
I20260809 16:33:24.240254 26391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:24.226643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:36689 (local address 127.30.23.129:54265)
0809 16:33:24.226815 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:24.226827 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:24.226891 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:24.227850 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:24.227854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:24.228562 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:24.228852 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:24.228860 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:24.230675 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:24.230676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:24.231995 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:24.231998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:24.232928 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:24.232929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:24.233099 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:24.233861 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:24.233887 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:24.236052 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:24.238318 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:24.238327 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:24.238330 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:24.238617 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:24.238900 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:24.238901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:24.238901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:24.239022 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:24.239535 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:24.239543 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:24.239694 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 16:33:24.239812 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 16:33:24.240111 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":213,"thread_start_us":68,"threads_started":1}
W20260809 16:33:24.241525 25747 leader_election.cc:341] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333): Not found: Tablet not found: 94d7c17b0fc94baab736e9faa455408e
W20260809 16:33:24.241667 25749 leader_election.cc:341] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689): Not found: Tablet not found: 94d7c17b0fc94baab736e9faa455408e
I20260809 16:33:24.241760 25749 leader_election.cc:302] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98cdb8a0a78c4179b6421b96d1658f9d; no voters: a6267fa1a6464ae6a3b1c47f7fbcc4d0, f07e6d7cde2d40c2bcedd2c2b4cc5694
I20260809 16:33:24.241881 26390 raft_consensus.cc:2726] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:33:24.990221 26395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:24.989931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:57141)
0809 16:33:24.990125 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:24.990171 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":77,"threads_started":1}
I20260809 16:33:25.064519 26396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:25.064221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:38307)
0809 16:33:25.064412 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:25.064468 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
I20260809 16:33:25.206983 26398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:25.206726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:50785)
0809 16:33:25.206896 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:25.206936 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260809 16:33:25.991075 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:25.990842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:35363)
0809 16:33:25.990987 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:25.991025 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260809 16:33:26.065403 26401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:26.065116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:43177)
0809 16:33:26.065274 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:26.065328 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260809 16:33:26.119542 26402 raft_consensus.cc:482] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:33:26.119652 26402 raft_consensus.cc:504] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:26.119849 26402 leader_election.cc:288] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689), a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333)
W20260809 16:33:26.120272 25747 leader_election.cc:341] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333): Not found: Tablet not found: 94d7c17b0fc94baab736e9faa455408e
W20260809 16:33:26.120391 25749 leader_election.cc:341] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689): Not found: Tablet not found: 94d7c17b0fc94baab736e9faa455408e
I20260809 16:33:26.120471 25749 leader_election.cc:302] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98cdb8a0a78c4179b6421b96d1658f9d; no voters: a6267fa1a6464ae6a3b1c47f7fbcc4d0, f07e6d7cde2d40c2bcedd2c2b4cc5694
I20260809 16:33:26.120550 26402 raft_consensus.cc:2726] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:33:26.207890 26403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:26.207579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:49111)
0809 16:33:26.207778 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:26.207823 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":95,"threads_started":1}
I20260809 16:33:26.991989 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:26.991700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:58789)
0809 16:33:26.991881 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:26.991935 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
I20260809 16:33:27.066247 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:27.065984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:44697)
0809 16:33:27.066149 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:27.066192 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
I20260809 16:33:27.208738 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:27.208476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:49881)
0809 16:33:27.208646 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:27.208690 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260809 16:33:27.993068 26408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:27.992758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:41215)
0809 16:33:27.992937 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:27.992997 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":84,"threads_started":1}
I20260809 16:33:28.067152 26411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:28.066874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:51555)
0809 16:33:28.067061 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:28.067105 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":107,"threads_started":1}
I20260809 16:33:28.209599 26412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:28.209293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:59993)
0809 16:33:28.209512 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:28.209551 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":137,"threads_started":1}
I20260809 16:33:28.610409 26413 raft_consensus.cc:482] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:33:28.610519 26413 raft_consensus.cc:504] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:28.610682 26413 leader_election.cc:288] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689), a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333)
W20260809 16:33:28.611205 25749 leader_election.cc:341] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689): Not found: Tablet not found: 94d7c17b0fc94baab736e9faa455408e
W20260809 16:33:28.611316 25747 leader_election.cc:341] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333): Not found: Tablet not found: 94d7c17b0fc94baab736e9faa455408e
I20260809 16:33:28.611495 25747 leader_election.cc:302] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98cdb8a0a78c4179b6421b96d1658f9d; no voters: a6267fa1a6464ae6a3b1c47f7fbcc4d0, f07e6d7cde2d40c2bcedd2c2b4cc5694
I20260809 16:33:28.611615 26413 raft_consensus.cc:2726] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:33:28.993947 26414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:28.993686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:42487)
0809 16:33:28.993848 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:28.993895 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260809 16:33:29.067962 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:29.067671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:46425)
0809 16:33:29.067865 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:29.067912 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":70,"threads_started":1}
I20260809 16:33:29.210515 26416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:29.210268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:56411)
0809 16:33:29.210428 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:29.210465 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":75,"threads_started":1}
I20260809 16:33:29.994696 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:29.994459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:45453)
0809 16:33:29.994608 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:29.994648 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260809 16:33:30.068886 26419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:30.068581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:49307)
0809 16:33:30.068768 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:30.068814 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":86,"threads_started":1}
I20260809 16:33:30.211349 26421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:30.211063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:57531)
0809 16:33:30.211239 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:30.211298 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":82,"threads_started":1}
I20260809 16:33:30.995711 26423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:30.995360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:46887)
0809 16:33:30.995610 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:30.995661 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":147,"threads_started":1}
I20260809 16:33:31.069738 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:31.069496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:45655)
0809 16:33:31.069644 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:31.069690 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260809 16:33:31.212124 26426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:31.211900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:59127)
0809 16:33:31.212033 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:31.212073 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260809 16:33:31.996505 26427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:31.996254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:49033)
0809 16:33:31.996416 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:31.996456 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260809 16:33:32.070504 26428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:32.070267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:59681)
0809 16:33:32.070421 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:32.070458 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260809 16:33:32.212883 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:32.212647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:34913)
0809 16:33:32.212797 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:32.212837 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260809 16:33:32.997537 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:32.997113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:38731)
0809 16:33:32.997282 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:32.997331 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":88,"threads_started":1}
I20260809 16:33:33.071426 26434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:33.071169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:57589)
0809 16:33:33.071318 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:33.071361 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260809 16:33:33.213793 26435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:33.213498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:58045)
0809 16:33:33.213669 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:33.213742 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":67,"threads_started":1}
I20260809 16:33:33.420832 26437 raft_consensus.cc:482] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:33:33.420946 26437 raft_consensus.cc:504] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07e6d7cde2d40c2bcedd2c2b4cc5694" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 36689 } } peers { permanent_uuid: "a6267fa1a6464ae6a3b1c47f7fbcc4d0" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 41333 } } peers { permanent_uuid: "98cdb8a0a78c4179b6421b96d1658f9d" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 45881 } }
I20260809 16:33:33.421135 26437 leader_election.cc:288] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689), a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333)
W20260809 16:33:33.421670 25747 leader_election.cc:341] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6267fa1a6464ae6a3b1c47f7fbcc4d0 (127.30.23.131:41333): Not found: Tablet not found: 94d7c17b0fc94baab736e9faa455408e
W20260809 16:33:33.421963 25749 leader_election.cc:341] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f07e6d7cde2d40c2bcedd2c2b4cc5694 (127.30.23.130:36689): Not found: Tablet not found: 94d7c17b0fc94baab736e9faa455408e
I20260809 16:33:33.422034 25749 leader_election.cc:302] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 98cdb8a0a78c4179b6421b96d1658f9d; no voters: a6267fa1a6464ae6a3b1c47f7fbcc4d0, f07e6d7cde2d40c2bcedd2c2b4cc5694
I20260809 16:33:33.422135 26437 raft_consensus.cc:2726] T 94d7c17b0fc94baab736e9faa455408e P 98cdb8a0a78c4179b6421b96d1658f9d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:33:33.998497 26439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:33.998178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:43049)
0809 16:33:33.998392 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:33.998441 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":95,"threads_started":1}
I20260809 16:33:34.072314 26440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:34.072048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:41271)
0809 16:33:34.072211 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:34.072259 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
I20260809 16:33:34.214942 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:34.214463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:60823)
0809 16:33:34.214722 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:34.214806 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":84,"threads_started":1}
I20260809 16:33:34.999493 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:34.999212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:54585)
0809 16:33:34.999395 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:34.999444 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":75,"threads_started":1}
I20260809 16:33:35.073264 26445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:35.073023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:45671)
0809 16:33:35.073159 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:35.073215 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260809 16:33:35.215979 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:35.215698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:58013)
0809 16:33:35.215883 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:35.215927 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":82,"threads_started":1}
I20260809 16:33:36.000484 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:36.000161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:44967)
0809 16:33:36.000385 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:36.000434 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":110,"threads_started":1}
I20260809 16:33:36.074172 26448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:36.073882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:48727)
0809 16:33:36.074078 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:36.074122 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":95,"threads_started":1}
I20260809 16:33:36.216964 26451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:36.216681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:57137)
0809 16:33:36.216864 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:36.216913 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":79,"threads_started":1}
I20260809 16:33:37.001608 26453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:37.001273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:37843)
0809 16:33:37.001499 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:37.001558 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":137,"threads_started":1}
I20260809 16:33:37.075117 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:37.074780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:33517)
0809 16:33:37.075002 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:37.075064 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":76,"threads_started":1}
I20260809 16:33:37.217921 26458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:37.217630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:58823)
0809 16:33:37.217812 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:37.217869 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
I20260809 16:33:38.002648 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:38.002373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.129:38805)
0809 16:33:38.002563 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:38.002602 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":98,"threads_started":1}
I20260809 16:33:38.076152 26462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:38.075907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.130:42761)
0809 16:33:38.076057 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:38.076104 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260809 16:33:38.216162 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 25672
I20260809 16:33:38.228575 26463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:38.222935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33381 (local address 127.30.23.131:49255)
0809 16:33:38.228428 (+ 5493us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:38.228499 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.190:33381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5463,"thread_start_us":1362,"threads_started":1}
I20260809 16:33:38.244408 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 25931
I20260809 16:33:38.259171 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 26164
2026-08-09T16:33:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58927 ms)
[----------] 30 tests from SecurityITest (254730 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:33:38 dist-test-slave-j2vl krb5kdc[26466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:33:38 dist-test-slave-j2vl krb5kdc[26466](info): set up 2 sockets
Aug 09 16:33:38 dist-test-slave-j2vl krb5kdc[26466](info): commencing operation
krb5kdc: starting...
W20260809 16:33:39.589640 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:33:39 dist-test-slave-j2vl krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:33:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:33:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:33:39.741714 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:43713
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:42909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:43713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:39.832966 26488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:39.833343 26488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:39.833420 26488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:39.834686 26488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:39.834785 26488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:39.834812 26488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:39.834844 26488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:39.834868 26488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:39.836484 26488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:42909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:43713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:43713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:39.836804 26488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:39.837067 26488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:39.842501 26496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:39.842525 26498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:39.842507 26503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:39.842989 26488 server_base.cc:1034] running on GCE node
I20260809 16:33:39.843287 26488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:39.843632 26488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:39.844816 26488 hybrid_clock.cc:648] HybridClock initialized: now 1786293219844795 us; error 41 us; skew 500 ppm
Aug 09 16:33:39 dist-test-slave-j2vl krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293219, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:39.848096 26488 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:33:39.848829 26488 webserver.cc:460] Webserver started at http://127.30.23.190:38097/ using document root <none> and password file <none>
I20260809 16:33:39.849185 26488 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:39.849314 26488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:39.849516 26488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:39.850683 26488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "80d51668e2e94b6b8b8c7feafb3a65c8"
format_stamp: "Formatted at 2026-08-09 16:33:39 on dist-test-slave-j2vl"
server_key: "54a1e18108e0f2d7e3caba47658d2184"
server_key_iv: "129acc546f764dc9aae37b1f97e16a4f"
server_key_version: "encryptionkey@0"
I20260809 16:33:39.851092 26488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "80d51668e2e94b6b8b8c7feafb3a65c8"
format_stamp: "Formatted at 2026-08-09 16:33:39 on dist-test-slave-j2vl"
server_key: "54a1e18108e0f2d7e3caba47658d2184"
server_key_iv: "129acc546f764dc9aae37b1f97e16a4f"
server_key_version: "encryptionkey@0"
I20260809 16:33:39.852663 26488 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:33:39.853729 26518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:39.854115 26488 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:39.854247 26488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "80d51668e2e94b6b8b8c7feafb3a65c8"
format_stamp: "Formatted at 2026-08-09 16:33:39 on dist-test-slave-j2vl"
server_key: "54a1e18108e0f2d7e3caba47658d2184"
server_key_iv: "129acc546f764dc9aae37b1f97e16a4f"
server_key_version: "encryptionkey@0"
I20260809 16:33:39.854362 26488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:39.868709 26488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:39.871603 26488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:39.871822 26488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:39.877812 26488 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:43713
I20260809 16:33:39.877897 26596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:43713 every 8 connection(s)
I20260809 16:33:39.878345 26488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:39.878952 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 26488
I20260809 16:33:39.879118 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:33:39.879366 30814 external_mini_cluster.cc:1428] Setting key 7e8bcbab22cad8fdc9e0906d4fa70bae
I20260809 16:33:39.879751 26600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:39.883090 26600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8: Bootstrap starting.
I20260809 16:33:39.883811 26600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:39.884462 26600 log.cc:824] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:39.885409 26600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8: No bootstrap required, opened a new log
Aug 09 16:33:39 dist-test-slave-j2vl krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:33:39.887825 26600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d51668e2e94b6b8b8c7feafb3a65c8" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43713 } }
I20260809 16:33:39.888031 26600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:39.888082 26600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80d51668e2e94b6b8b8c7feafb3a65c8, State: Initialized, Role: FOLLOWER
I20260809 16:33:39.888265 26600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d51668e2e94b6b8b8c7feafb3a65c8" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43713 } }
I20260809 16:33:39.888370 26600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:39.888408 26600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:39.888458 26600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:39.889199 26600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d51668e2e94b6b8b8c7feafb3a65c8" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43713 } }
I20260809 16:33:39.889374 26600 leader_election.cc:302] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80d51668e2e94b6b8b8c7feafb3a65c8; no voters:
I20260809 16:33:39.889923 26600 leader_election.cc:288] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:33:39.889988 26605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:39.890213 26605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [term 1 LEADER]: Becoming Leader. State: Replica: 80d51668e2e94b6b8b8c7feafb3a65c8, State: Running, Role: LEADER
I20260809 16:33:39.890394 26600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:39.890547 26605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d51668e2e94b6b8b8c7feafb3a65c8" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43713 } }
I20260809 16:33:39.891031 26605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80d51668e2e94b6b8b8c7feafb3a65c8. Latest consensus state: current_term: 1 leader_uuid: "80d51668e2e94b6b8b8c7feafb3a65c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d51668e2e94b6b8b8c7feafb3a65c8" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43713 } } }
I20260809 16:33:39.891310 26605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:39.891217 26607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80d51668e2e94b6b8b8c7feafb3a65c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d51668e2e94b6b8b8c7feafb3a65c8" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43713 } } }
I20260809 16:33:39.891796 26607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:39.891783 26618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:39.892577 26618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:39.892961 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:39.879959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45586 (local address 127.30.23.190:43713)
0809 16:33:39.880183 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:39.880193 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:39.880231 (+ 38us) server_negotiation.cc:407] Connection header received
0809 16:33:39.880966 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:39.880975 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:39.881031 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:39.881203 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:39.882058 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:39.882999 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:39.883852 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:39.884348 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:39.884477 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:39.887812 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:39.887817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:39.887817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:39.887831 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:39.890345 (+ 2514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:39.891000 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:39.891004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:39.891004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:39.891146 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:39.891474 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:39.891474 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:39.891475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:39.891870 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:39.892480 (+ 610us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:39.892562 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 16:33:39.892732 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":91,"threads_started":1}
I20260809 16:33:39.894372 26618 catalog_manager.cc:1347] Generated new cluster ID: adced7b21b044212811ae96219ca37e2
I20260809 16:33:39.894433 26618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:33:39.908434 26618 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:33:39.909271 26618 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:39.917694 26618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8: Generated new TSK 0
I20260809 16:33:39.918406 26618 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:33:39.962838 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43713
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:42909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:40.061529 26641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:40.061811 26641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:40.061861 26641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:40.063197 26641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:40.063284 26641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:40.063337 26641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:33:40.065035 26641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:42909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:40.065369 26641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:40.065662 26641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:33:40.072512 26641 server_base.cc:1034] running on GCE node
W20260809 16:33:40.072410 26654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:40.072410 26649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:40.072440 26656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:40.073001 26641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:40.073288 26641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:40.074498 26641 hybrid_clock.cc:648] HybridClock initialized: now 1786293220074484 us; error 30 us; skew 500 ppm
Aug 09 16:33:40 dist-test-slave-j2vl krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293220, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:40.078058 26641 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:33:40.078650 26641 webserver.cc:460] Webserver started at http://127.30.23.129:41759/ using document root <none> and password file <none>
I20260809 16:33:40.078935 26641 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:40.079018 26641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:40.079198 26641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:40.080427 26641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "dc0645d71e3d450a989391f286354b21"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "3d1fd57547eb0e7506c7eee0d50f4981"
server_key_iv: "b85631d896321668171d6c8175d67543"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.080858 26641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "dc0645d71e3d450a989391f286354b21"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "3d1fd57547eb0e7506c7eee0d50f4981"
server_key_iv: "b85631d896321668171d6c8175d67543"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.082526 26641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 16:33:40.083460 26670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:40.083720 26641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:40.083878 26641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "dc0645d71e3d450a989391f286354b21"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "3d1fd57547eb0e7506c7eee0d50f4981"
server_key_iv: "b85631d896321668171d6c8175d67543"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.084149 26641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:40.098706 26641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:40.101755 26641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:40.101953 26641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:40.102242 26641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:40.102618 26641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:40.102679 26641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:40.102721 26641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:40.102775 26641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:40.109181 26641 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:37929
I20260809 16:33:40.109232 26840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:37929 every 8 connection(s)
I20260809 16:33:40.109786 26641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:33:40.110450 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 26641
I20260809 16:33:40.110577 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:33:40.110807 30814 external_mini_cluster.cc:1428] Setting key 1735ff5f6dc1245f2cedc4caff2563ab
Aug 09 16:33:40 dist-test-slave-j2vl krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293220, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
I20260809 16:33:40.122718 26845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.110995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:43713 (local address 127.30.23.129:50093)
0809 16:33:40.111212 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:40.111276 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:40.112003 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:40.112333 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:40.112337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:40.112961 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:40.113170 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.113175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.114230 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.114232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.115619 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.115622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.116080 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.116081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.116243 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.116932 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:40.116945 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:40.118898 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:40.120970 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:40.120987 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:40.120990 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:40.121278 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:40.121569 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:40.121570 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:40.121570 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:40.121701 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:40.122026 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:40.122034 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:40.122194 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 16:33:40.122308 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 16:33:40.122462 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":116,"threads_started":1}
I20260809 16:33:40.122773 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.111085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:50093 (local address 127.30.23.190:43713)
0809 16:33:40.111149 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:40.111151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:40.111374 (+ 223us) server_negotiation.cc:407] Connection header received
0809 16:33:40.112148 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:40.112150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:40.112163 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:40.112242 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:40.113281 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.114114 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.115750 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.115967 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.116045 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.119020 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:40.119022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:40.119022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:40.119039 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:40.120843 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:40.121404 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:40.121406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:40.121406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:40.121482 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:40.121789 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:40.121795 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:40.121795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:40.121923 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:40.121984 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:40.122374 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 16:33:40.122553 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:33:40.123409 26843 heartbeater.cc:347] Connected to a master server at 127.30.23.190:43713
I20260809 16:33:40.123524 26843 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:40.123749 26843 heartbeater.cc:511] Master 127.30.23.190:43713 requested a full tablet report, sending...
I20260809 16:33:40.124372 26544 ts_manager.cc:194] Registered new tserver with Master: dc0645d71e3d450a989391f286354b21 (127.30.23.129:37929)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
I20260809 16:33:40.125191 26544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:50093
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:33:40.146337 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43713
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:42909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:40.234032 26849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:40.234263 26849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:40.234288 26849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:40.235433 26849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:40.235476 26849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:40.235502 26849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:33:40.236958 26849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:42909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:40.237169 26849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:40.237375 26849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:40.243041 26861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:40.243088 26849 server_base.cc:1034] running on GCE node
W20260809 16:33:40.243058 26863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:40.243050 26859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:40.243487 26849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:40.243742 26849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:40.244899 26849 hybrid_clock.cc:648] HybridClock initialized: now 1786293220244889 us; error 34 us; skew 500 ppm
Aug 09 16:33:40 dist-test-slave-j2vl krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293220, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:40.247887 26849 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:33:40.248438 26849 webserver.cc:460] Webserver started at http://127.30.23.130:42541/ using document root <none> and password file <none>
I20260809 16:33:40.248703 26849 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:40.248754 26849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:40.248896 26849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:40.250125 26849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "4220e112bab18e3dc80c299556e7b679"
server_key_iv: "142ebfbbc42fed6aa1653eb42a758873"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.250515 26849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "4220e112bab18e3dc80c299556e7b679"
server_key_iv: "142ebfbbc42fed6aa1653eb42a758873"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.252175 26849 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:33:40.253131 26874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:40.253424 26849 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:40.253533 26849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "4220e112bab18e3dc80c299556e7b679"
server_key_iv: "142ebfbbc42fed6aa1653eb42a758873"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.253623 26849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:40.262697 26849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:40.265058 26849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:40.265251 26849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:40.265571 26849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:40.265937 26849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:40.265998 26849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:40.266047 26849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:40.266098 26849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:40.272830 26849 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:40271
I20260809 16:33:40.272881 27063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:40271 every 8 connection(s)
I20260809 16:33:40.273283 26849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:33:40 dist-test-slave-j2vl krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293220, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:33:40.283473 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.274187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:33015 (local address 127.30.23.190:43713)
0809 16:33:40.274278 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:40.274280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:40.274402 (+ 122us) server_negotiation.cc:407] Connection header received
0809 16:33:40.274952 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:40.274953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:40.274965 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:40.275025 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:40.276082 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.276603 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.277821 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.278021 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.278086 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.280519 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:40.280521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:40.280521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:40.280530 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:40.281867 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:40.282332 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:40.282333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:40.282333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:40.282436 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:40.282721 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:40.282723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:40.282723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:40.282861 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:40.282934 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:40.283321 (+ 387us) server_negotiation.cc:299] Negotiation successful
0809 16:33:40.283396 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 16:33:40.283500 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 26849
I20260809 16:33:40.283600 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:33:40.283842 30814 external_mini_cluster.cc:1428] Setting key 680acb38909ba417e22603bf7ccd9c53
I20260809 16:33:40.283859 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.274098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:43713 (local address 127.30.23.130:33015)
0809 16:33:40.274270 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:40.274315 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:40.274837 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:40.275146 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:40.275150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:40.275763 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:40.275966 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.275970 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.276707 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.276708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.277711 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.277714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.278107 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.278108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.278218 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.278805 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:40.278818 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:40.280404 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:40.281966 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:40.281973 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:40.281976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:40.282225 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:40.282514 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:40.282515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:40.282515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:40.282635 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:40.282957 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:40.282964 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:40.283119 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 16:33:40.283303 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 16:33:40.283539 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260809 16:33:40.284703 27066 heartbeater.cc:347] Connected to a master server at 127.30.23.190:43713
I20260809 16:33:40.284821 27066 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:40.285050 27066 heartbeater.cc:511] Master 127.30.23.190:43713 requested a full tablet report, sending...
I20260809 16:33:40.285619 26544 ts_manager.cc:194] Registered new tserver with Master: 3dfdaf3fe5084a9d8615b1d5b0f7e411 (127.30.23.130:40271)
I20260809 16:33:40.286145 26544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:33015
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:33:40.318104 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43713
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:42909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:40.407956 27071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:40.408246 27071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:40.408288 27071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:40.409754 27071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:40.409860 27071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:40.409914 27071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:33:40.411691 27071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:42909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:40.412009 27071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:40.412268 27071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:40.417891 27077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:40.418002 27076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:40.418088 27071 server_base.cc:1034] running on GCE node
W20260809 16:33:40.418196 27079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:40.418424 27071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:40.418761 27071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:40.419957 27071 hybrid_clock.cc:648] HybridClock initialized: now 1786293220419944 us; error 29 us; skew 500 ppm
Aug 09 16:33:40 dist-test-slave-j2vl krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293220, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:40.422816 27071 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:33:40.423280 27071 webserver.cc:460] Webserver started at http://127.30.23.131:41639/ using document root <none> and password file <none>
I20260809 16:33:40.423513 27071 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:40.423561 27071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:40.423697 27071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:40.424834 27071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "55acde51a7824cdcb6ee5619bc27181d"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "8c442269029b67dbac10240bd9a0ed55"
server_key_iv: "dc19d220481fcb38a73aeeaefaaacc0b"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.425220 27071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "55acde51a7824cdcb6ee5619bc27181d"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "8c442269029b67dbac10240bd9a0ed55"
server_key_iv: "dc19d220481fcb38a73aeeaefaaacc0b"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.427067 27071 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260809 16:33:40.428130 27090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:40.428498 27071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:40.428599 27071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "55acde51a7824cdcb6ee5619bc27181d"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "8c442269029b67dbac10240bd9a0ed55"
server_key_iv: "dc19d220481fcb38a73aeeaefaaacc0b"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.428720 27071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:40.452152 27071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:40.454526 27071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:40.454659 27071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:40.454912 27071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:40.455305 27071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:40.455367 27071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:40.455410 27071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:40.455459 27071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:40.463753 27071 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:35199
I20260809 16:33:40.463835 27269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:35199 every 8 connection(s)
I20260809 16:33:40.464326 27071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:33:40.465445 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 27071
I20260809 16:33:40.465559 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:33:40.465801 30814 external_mini_cluster.cc:1428] Setting key a66e084328b14df1863a0e21f38ac77f
Aug 09 16:33:40 dist-test-slave-j2vl krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293220, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:33:40.476979 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.465948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:53101 (local address 127.30.23.190:43713)
0809 16:33:40.466001 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:40.466004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:40.466876 (+ 872us) server_negotiation.cc:407] Connection header received
0809 16:33:40.467602 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:40.467603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:40.467617 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:40.467707 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:40.469234 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.469865 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.470931 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.471139 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.471214 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.473811 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:40.473812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:40.473813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:40.473821 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:40.475316 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:40.475811 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:40.475812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:40.475813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:40.475879 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:40.476180 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:40.476180 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:40.476180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:40.476300 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:40.476358 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:40.476751 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 16:33:40.476911 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:33:40.477012 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.465753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:43713 (local address 127.30.23.131:53101)
0809 16:33:40.466695 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:40.466784 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:40.467501 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:40.467788 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:40.467792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:40.468660 (+ 868us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:40.468954 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.468960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.469997 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.469998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.470829 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.470831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.471258 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.471259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.471418 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.472042 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:40.472051 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:40.473674 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:40.475412 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:40.475420 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:40.475423 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:40.475696 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:40.475981 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:40.475982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:40.475982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:40.476110 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:40.476390 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:40.476397 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:40.476544 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 16:33:40.476641 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 16:33:40.476824 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"thread_start_us":105,"threads_started":1}
I20260809 16:33:40.477713 27270 heartbeater.cc:347] Connected to a master server at 127.30.23.190:43713
I20260809 16:33:40.477849 27270 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:40.478073 27270 heartbeater.cc:511] Master 127.30.23.190:43713 requested a full tablet report, sending...
I20260809 16:33:40.478612 26543 ts_manager.cc:194] Registered new tserver with Master: 55acde51a7824cdcb6ee5619bc27181d (127.30.23.131:35199)
I20260809 16:33:40.479007 26543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:53101
I20260809 16:33:40.489593 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:33:40.497728 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.491390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.30.23.190:43713)
0809 16:33:40.491446 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:40.491448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:40.491481 (+ 33us) server_negotiation.cc:407] Connection header received
0809 16:33:40.491553 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:40.491554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:40.491564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:40.491621 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:40.492251 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.492855 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.493631 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.493839 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.493907 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.495171 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:40.495172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:40.495172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:40.495179 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:40.496551 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:40.496922 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:40.496923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:40.496923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:40.496977 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:40.497156 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:40.497156 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:40.497156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:40.497302 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:40.497459 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:40.497588 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 16:33:40.497661 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:33:40.499876 26543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:33:40.501533 26543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:33:40.508600 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.504275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:40271 (local address 127.0.0.1:44104)
0809 16:33:40.504682 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:40.504732 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:40.504828 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:40.505179 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:40.505182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:40.505184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:40.505273 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.505275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.506317 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.506319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.507430 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.507433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.508137 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.508138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.508232 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.508246 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:33:40.508350 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 16:33:40.508503 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":86,"threads_started":1}
I20260809 16:33:40.508605 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.504465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44104 (local address 127.30.23.130:40271)
0809 16:33:40.504648 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:40.504653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:40.504926 (+ 273us) server_negotiation.cc:407] Connection header received
0809 16:33:40.504976 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:40.504979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:40.504998 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:40.505074 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:40.505428 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.506213 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.507517 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.508049 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.508137 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.508201 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:40.508450 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 16:33:40.508502 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260809 16:33:40.508913 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.504540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:37929 (local address 127.0.0.1:47880)
0809 16:33:40.505089 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:40.505098 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:40.505137 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:40.505429 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:40.505430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:40.505433 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:40.505508 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.505510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.506852 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.506853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.507711 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.507714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.508663 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.508664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.508731 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.508735 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:40.508778 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:33:40.508826 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":80,"threads_started":1}
I20260809 16:33:40.509172 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.504670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47880 (local address 127.30.23.129:37929)
0809 16:33:40.505012 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:40.505016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:40.505234 (+ 218us) server_negotiation.cc:407] Connection header received
0809 16:33:40.505276 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:40.505278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:40.505294 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:40.505397 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:40.505591 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.506701 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.507805 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.508514 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.508583 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.508628 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:40.508993 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 16:33:40.509071 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":93,"threads_started":1}
I20260809 16:33:40.509994 26959 tablet_service.cc:1467] Processing CreateTablet for tablet de39dad614a8443f885c7095538be68b (DEFAULT_TABLE table=test-table [id=b0d06d0121a445e3a0977c4aa01e8418]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:40.510440 26959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de39dad614a8443f885c7095538be68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:40.510468 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.504692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56756 (local address 127.30.23.131:35199)
0809 16:33:40.504889 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:40.504894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:40.504986 (+ 92us) server_negotiation.cc:407] Connection header received
0809 16:33:40.505066 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:40.505068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:40.505081 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:40.505158 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:40.505525 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.506701 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.508964 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.510009 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.510162 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.510257 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:40.510365 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 16:33:40.510402 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260809 16:33:40.510501 27288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.504275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:35199 (local address 127.0.0.1:56756)
0809 16:33:40.504925 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:40.504937 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:40.504999 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:40.505404 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:40.505406 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:40.505408 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:40.505465 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.505467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.507827 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.507828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.508881 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.508883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.510192 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.510193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.510267 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.510271 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:40.510343 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:33:40.510412 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260809 16:33:40.511173 27171 tablet_service.cc:1467] Processing CreateTablet for tablet de39dad614a8443f885c7095538be68b (DEFAULT_TABLE table=test-table [id=b0d06d0121a445e3a0977c4aa01e8418]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:40.511507 27171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de39dad614a8443f885c7095538be68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:40.513543 27301 tablet_bootstrap.cc:492] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411: Bootstrap starting.
I20260809 16:33:40.514008 26736 tablet_service.cc:1467] Processing CreateTablet for tablet de39dad614a8443f885c7095538be68b (DEFAULT_TABLE table=test-table [id=b0d06d0121a445e3a0977c4aa01e8418]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:40.514367 27301 tablet_bootstrap.cc:654] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:40.514380 26736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de39dad614a8443f885c7095538be68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:40.514496 27302 tablet_bootstrap.cc:492] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d: Bootstrap starting.
I20260809 16:33:40.515165 27302 tablet_bootstrap.cc:654] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:40.515141 27301 log.cc:824] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:40.515774 27302 log.cc:824] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:40.516218 27301 tablet_bootstrap.cc:492] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411: No bootstrap required, opened a new log
I20260809 16:33:40.516335 27301 ts_tablet_manager.cc:1397] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:33:40.517933 27309 tablet_bootstrap.cc:492] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21: Bootstrap starting.
I20260809 16:33:40.518054 27302 tablet_bootstrap.cc:492] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d: No bootstrap required, opened a new log
I20260809 16:33:40.518129 27302 ts_tablet_manager.cc:1397] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 16:33:40.518481 27301 raft_consensus.cc:348] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40271 } } peers { permanent_uuid: "55acde51a7824cdcb6ee5619bc27181d" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35199 } } peers { permanent_uuid: "dc0645d71e3d450a989391f286354b21" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37929 } }
I20260809 16:33:40.518669 27301 raft_consensus.cc:374] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:40.518733 27309 tablet_bootstrap.cc:654] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:40.518733 27301 raft_consensus.cc:729] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dfdaf3fe5084a9d8615b1d5b0f7e411, State: Initialized, Role: FOLLOWER
I20260809 16:33:40.518877 27301 consensus_queue.cc:260] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40271 } } peers { permanent_uuid: "55acde51a7824cdcb6ee5619bc27181d" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35199 } } peers { permanent_uuid: "dc0645d71e3d450a989391f286354b21" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37929 } }
I20260809 16:33:40.519209 27301 ts_tablet_manager.cc:1428] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:33:40.519209 27066 heartbeater.cc:503] Master 127.30.23.190:43713 was elected leader, sending a full tablet report...
I20260809 16:33:40.519304 27309 log.cc:824] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:40.520135 27309 tablet_bootstrap.cc:492] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21: No bootstrap required, opened a new log
I20260809 16:33:40.520229 27309 ts_tablet_manager.cc:1397] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:33:40.520362 27302 raft_consensus.cc:348] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40271 } } peers { permanent_uuid: "55acde51a7824cdcb6ee5619bc27181d" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35199 } } peers { permanent_uuid: "dc0645d71e3d450a989391f286354b21" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37929 } }
I20260809 16:33:40.520697 27302 raft_consensus.cc:374] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:40.520764 27302 raft_consensus.cc:729] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55acde51a7824cdcb6ee5619bc27181d, State: Initialized, Role: FOLLOWER
I20260809 16:33:40.520954 27302 consensus_queue.cc:260] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40271 } } peers { permanent_uuid: "55acde51a7824cdcb6ee5619bc27181d" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35199 } } peers { permanent_uuid: "dc0645d71e3d450a989391f286354b21" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37929 } }
I20260809 16:33:40.521296 27302 ts_tablet_manager.cc:1428] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:33:40.521425 27270 heartbeater.cc:503] Master 127.30.23.190:43713 was elected leader, sending a full tablet report...
I20260809 16:33:40.522181 27309 raft_consensus.cc:348] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40271 } } peers { permanent_uuid: "55acde51a7824cdcb6ee5619bc27181d" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35199 } } peers { permanent_uuid: "dc0645d71e3d450a989391f286354b21" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37929 } }
I20260809 16:33:40.522342 27309 raft_consensus.cc:374] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:40.522390 27309 raft_consensus.cc:729] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0645d71e3d450a989391f286354b21, State: Initialized, Role: FOLLOWER
I20260809 16:33:40.522511 27309 consensus_queue.cc:260] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40271 } } peers { permanent_uuid: "55acde51a7824cdcb6ee5619bc27181d" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35199 } } peers { permanent_uuid: "dc0645d71e3d450a989391f286354b21" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37929 } }
I20260809 16:33:40.522821 27309 ts_tablet_manager.cc:1428] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:33:40.522835 26843 heartbeater.cc:503] Master 127.30.23.190:43713 was elected leader, sending a full tablet report...
W20260809 16:33:40.523929 27067 tablet.cc:2369] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:33:40.524274 27314 raft_consensus.cc:482] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:33:40.524387 27314 raft_consensus.cc:504] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40271 } } peers { permanent_uuid: "55acde51a7824cdcb6ee5619bc27181d" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35199 } } peers { permanent_uuid: "dc0645d71e3d450a989391f286354b21" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37929 } }
I20260809 16:33:40.524696 27314 leader_election.cc:288] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3dfdaf3fe5084a9d8615b1d5b0f7e411 (127.30.23.130:40271), dc0645d71e3d450a989391f286354b21 (127.30.23.129:37929)
I20260809 16:33:40.528831 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.525096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:42045 (local address 127.30.23.129:37929)
0809 16:33:40.525143 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:40.525145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:40.525246 (+ 101us) server_negotiation.cc:407] Connection header received
0809 16:33:40.525309 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:40.525310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:40.525318 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:40.525594 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:40.525728 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.526417 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.527746 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.528357 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.528450 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.528460 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:40.528703 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 16:33:40.528752 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:33:40.528821 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.525025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:48181 (local address 127.30.23.130:40271)
0809 16:33:40.525074 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:40.525076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:40.525103 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:33:40.525127 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:40.525128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:40.525136 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:40.525183 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:40.525513 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.526329 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.527746 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.528364 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.528451 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.528460 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:40.528703 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 16:33:40.528752 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:33:40.528843 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.524915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:40271 (local address 127.30.23.131:48181)
0809 16:33:40.524994 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:40.525010 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:40.525066 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:40.525238 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:40.525239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:40.525240 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:40.525313 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.525315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.526403 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.526404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.527634 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.527637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.528482 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.528483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.528585 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.528587 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:33:40.528654 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:33:40.528727 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":42}
I20260809 16:33:40.528855 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.525042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:37929 (local address 127.30.23.131:42045)
0809 16:33:40.525194 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:40.525203 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:40.525254 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:40.525597 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:40.525598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:40.525599 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:40.525660 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.525661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.526493 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.526494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.527634 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.527637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.528484 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.528485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.528575 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.528581 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:33:40.528654 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 16:33:40.528727 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":8,"thread_start_us":110,"threads_started":1}
I20260809 16:33:40.529295 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de39dad614a8443f885c7095538be68b" candidate_uuid: "55acde51a7824cdcb6ee5619bc27181d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411" is_pre_election: true
I20260809 16:33:40.529657 26989 raft_consensus.cc:2454] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55acde51a7824cdcb6ee5619bc27181d in term 0.
I20260809 16:33:40.529613 26764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de39dad614a8443f885c7095538be68b" candidate_uuid: "55acde51a7824cdcb6ee5619bc27181d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0645d71e3d450a989391f286354b21" is_pre_election: true
I20260809 16:33:40.529755 26764 raft_consensus.cc:2454] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55acde51a7824cdcb6ee5619bc27181d in term 0.
I20260809 16:33:40.530001 27094 leader_election.cc:302] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55acde51a7824cdcb6ee5619bc27181d, dc0645d71e3d450a989391f286354b21; no voters:
I20260809 16:33:40.530139 27314 raft_consensus.cc:2781] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:33:40.530200 27314 raft_consensus.cc:482] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:33:40.530225 27314 raft_consensus.cc:3037] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:40.531055 27314 raft_consensus.cc:504] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40271 } } peers { permanent_uuid: "55acde51a7824cdcb6ee5619bc27181d" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35199 } } peers { permanent_uuid: "dc0645d71e3d450a989391f286354b21" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37929 } }
I20260809 16:33:40.531210 27314 leader_election.cc:288] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [CANDIDATE]: Term 1 election: Requested vote from peers 3dfdaf3fe5084a9d8615b1d5b0f7e411 (127.30.23.130:40271), dc0645d71e3d450a989391f286354b21 (127.30.23.129:37929)
I20260809 16:33:40.531435 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de39dad614a8443f885c7095538be68b" candidate_uuid: "55acde51a7824cdcb6ee5619bc27181d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411"
I20260809 16:33:40.531431 26764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de39dad614a8443f885c7095538be68b" candidate_uuid: "55acde51a7824cdcb6ee5619bc27181d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0645d71e3d450a989391f286354b21"
I20260809 16:33:40.531517 26764 raft_consensus.cc:3037] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:40.531538 26989 raft_consensus.cc:3037] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:40.532276 26989 raft_consensus.cc:2454] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55acde51a7824cdcb6ee5619bc27181d in term 1.
I20260809 16:33:40.532294 26764 raft_consensus.cc:2454] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55acde51a7824cdcb6ee5619bc27181d in term 1.
I20260809 16:33:40.532488 27094 leader_election.cc:302] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dfdaf3fe5084a9d8615b1d5b0f7e411, 55acde51a7824cdcb6ee5619bc27181d; no voters:
I20260809 16:33:40.532608 27314 raft_consensus.cc:2781] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:40.532812 27314 raft_consensus.cc:686] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [term 1 LEADER]: Becoming Leader. State: Replica: 55acde51a7824cdcb6ee5619bc27181d, State: Running, Role: LEADER
I20260809 16:33:40.532908 27314 consensus_queue.cc:237] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40271 } } peers { permanent_uuid: "55acde51a7824cdcb6ee5619bc27181d" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35199 } } peers { permanent_uuid: "dc0645d71e3d450a989391f286354b21" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37929 } }
I20260809 16:33:40.533747 26543 catalog_manager.cc:5495] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d reported cstate change: term changed from 0 to 1, leader changed from <none> to 55acde51a7824cdcb6ee5619bc27181d (127.30.23.131). New cstate: current_term: 1 leader_uuid: "55acde51a7824cdcb6ee5619bc27181d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55acde51a7824cdcb6ee5619bc27181d" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 35199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc0645d71e3d450a989391f286354b21" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37929 } health_report { overall_health: UNKNOWN } } }
I20260809 16:33:40.545573 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.543059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56760 (local address 127.30.23.131:35199)
0809 16:33:40.543125 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:40.543128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:40.543136 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:33:40.543182 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:40.543183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:40.543201 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:40.543280 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:33:40.543444 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.543940 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.544720 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.544930 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.545009 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.545187 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:33:40.545297 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:33:40.545434 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:40.545462 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 16:33:40.545484 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:33:40.547577 26989 raft_consensus.cc:1261] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411 [term 1 FOLLOWER]: Refusing update from remote peer 55acde51a7824cdcb6ee5619bc27181d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:40.547598 26764 raft_consensus.cc:1261] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21 [term 1 FOLLOWER]: Refusing update from remote peer 55acde51a7824cdcb6ee5619bc27181d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:40.547943 27314 consensus_queue.cc:1035] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dfdaf3fe5084a9d8615b1d5b0f7e411" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:40.548132 27324 consensus_queue.cc:1035] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0645d71e3d450a989391f286354b21" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:40.550491 27329 mvcc.cc:205] Tried to move back new op lower bound from 7316657031359684608 to 7316657031303606272. Current Snapshot: MvccSnapshot[applied={T|T < 7316657031359684608}]
I20260809 16:33:40.551192 27332 mvcc.cc:205] Tried to move back new op lower bound from 7316657031359684608 to 7316657031303606272. Current Snapshot: MvccSnapshot[applied={T|T < 7316657031359684608}]
I20260809 16:33:40.552791 26544 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:40.552906 26544 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:40.553931 26544 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80d51668e2e94b6b8b8c7feafb3a65c8: Sending DeleteTablet for 3 replicas of tablet de39dad614a8443f885c7095538be68b
I20260809 16:33:40.554368 26959 tablet_service.cc:1514] Processing DeleteTablet for tablet de39dad614a8443f885c7095538be68b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:40 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:44104
I20260809 16:33:40.554388 27171 tablet_service.cc:1514] Processing DeleteTablet for tablet de39dad614a8443f885c7095538be68b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:40 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:56756
I20260809 16:33:40.554649 26736 tablet_service.cc:1514] Processing DeleteTablet for tablet de39dad614a8443f885c7095538be68b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:40 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:47880
I20260809 16:33:40.554943 27352 tablet_replica.cc:330] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21: stopping tablet replica
I20260809 16:33:40.554961 27347 tablet_replica.cc:330] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411: stopping tablet replica
I20260809 16:33:40.555096 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 26641
I20260809 16:33:40.555109 27352 raft_consensus.cc:2229] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:40.555224 27352 raft_consensus.cc:2258] T de39dad614a8443f885c7095538be68b P dc0645d71e3d450a989391f286354b21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:40.563042 27348 tablet_replica.cc:330] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d: stopping tablet replica
I20260809 16:33:40.563218 27348 raft_consensus.cc:2229] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:33:40.563352 27348 raft_consensus.cc:2258] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:40.555066 27347 raft_consensus.cc:2229] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:40.563668 27347 raft_consensus.cc:2258] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:40.564172 27347 ts_tablet_manager.cc:1905] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:40.565935 27348 ts_tablet_manager.cc:1905] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:40.566005 27347 ts_tablet_manager.cc:1918] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:40.566111 27347 log.cc:1196] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/de39dad614a8443f885c7095538be68b
I20260809 16:33:40.566449 27347 ts_tablet_manager.cc:1939] T de39dad614a8443f885c7095538be68b P 3dfdaf3fe5084a9d8615b1d5b0f7e411: Deleting consensus metadata
I20260809 16:33:40.567924 27348 ts_tablet_manager.cc:1918] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:40.568082 27348 log.cc:1196] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/de39dad614a8443f885c7095538be68b
I20260809 16:33:40.568162 26523 catalog_manager.cc:4841] TS 3dfdaf3fe5084a9d8615b1d5b0f7e411 (127.30.23.130:40271): tablet de39dad614a8443f885c7095538be68b (table test-table [id=b0d06d0121a445e3a0977c4aa01e8418]) successfully deleted
I20260809 16:33:40.568457 27348 ts_tablet_manager.cc:1939] T de39dad614a8443f885c7095538be68b P 55acde51a7824cdcb6ee5619bc27181d: Deleting consensus metadata
W20260809 16:33:40.568879 26523 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:37929 (error 108)
I20260809 16:33:40.568929 26522 catalog_manager.cc:4841] TS 55acde51a7824cdcb6ee5619bc27181d (127.30.23.131:35199): tablet de39dad614a8443f885c7095538be68b (table test-table [id=b0d06d0121a445e3a0977c4aa01e8418]) successfully deleted
I20260809 16:33:40.569169 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 26849
I20260809 16:33:40.572026 27288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.569315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:37929 (local address 127.0.0.1:47890)
0809 16:33:40.571913 (+ 2598us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:40.571978 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:37929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2569}
W20260809 16:33:40.572204 26523 catalog_manager.cc:4584] TS dc0645d71e3d450a989391f286354b21 (127.30.23.129:37929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet de39dad614a8443f885c7095538be68b: Network error: Client connection negotiation failed: client connection to 127.30.23.129:37929: connect: Connection refused (error 111)
I20260809 16:33:40.577023 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 27071
I20260809 16:33:40.585011 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 26488
2026-08-09T16:33:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2321 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T16:33:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:33:40Z Disabled control of system clock
I20260809 16:33:40.616124 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:41935
--webserver_interface=127.30.23.190
--webserver_port=0
--builtin_ntp_servers=127.30.23.148:43187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:41935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:33:40.702454 27364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:40.702680 27364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:40.702702 27364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:40.703804 27364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:40.703846 27364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:40.703859 27364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:40.703873 27364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:40.703886 27364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:40.705196 27364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:41935
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:41935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:40.705490 27364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:40.705758 27364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:40.710915 27378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:40.710932 27375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:40.711047 27364 server_base.cc:1034] running on GCE node
W20260809 16:33:40.710942 27374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:40.711578 27364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:40.711862 27364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:40.713029 27364 hybrid_clock.cc:648] HybridClock initialized: now 1786293220713012 us; error 31 us; skew 500 ppm
I20260809 16:33:40.714421 27364 webserver.cc:460] Webserver started at http://127.30.23.190:37199/ using document root <none> and password file <none>
I20260809 16:33:40.714723 27364 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:40.714780 27364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:40.714927 27364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:40.716050 27364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "22d24bba8b0640e48aab24f27933ded3"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "295a07cc788f0be7b697d847389346aa"
server_key_iv: "e8eee29fbe9372b1c440ac5635ef218c"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.716430 27364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "22d24bba8b0640e48aab24f27933ded3"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "295a07cc788f0be7b697d847389346aa"
server_key_iv: "e8eee29fbe9372b1c440ac5635ef218c"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.718003 27364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:33:40.718883 27386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:40.719065 27364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:40.719182 27364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "22d24bba8b0640e48aab24f27933ded3"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "295a07cc788f0be7b697d847389346aa"
server_key_iv: "e8eee29fbe9372b1c440ac5635ef218c"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.719261 27364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:40.745683 27364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:40.746064 27364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:40.746237 27364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:40.750746 27364 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:41935
I20260809 16:33:40.750782 27469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:41935 every 8 connection(s)
I20260809 16:33:40.751168 27364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:40.751856 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 27364
I20260809 16:33:40.751986 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:33:40.752235 30814 external_mini_cluster.cc:1428] Setting key 03702de652a521cd9cbdf26d12b96c80
I20260809 16:33:40.752529 27475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:40.755884 27475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3: Bootstrap starting.
I20260809 16:33:40.756536 27475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:40.756920 27475 log.cc:824] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:40.757764 27475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3: No bootstrap required, opened a new log
I20260809 16:33:40.758064 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.752946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50020 (local address 127.30.23.190:41935)
0809 16:33:40.753167 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:40.753174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:40.753199 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:33:40.753780 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:40.753788 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:40.753854 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:40.754009 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:40.754243 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.755127 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.755918 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.756372 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.756522 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.757310 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:40.757314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:40.757314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:40.757539 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:40.757573 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:40.757584 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:40.757659 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:40.757748 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 16:33:40.757860 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
I20260809 16:33:40.759963 27475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d24bba8b0640e48aab24f27933ded3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41935 } }
I20260809 16:33:40.760139 27475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:40.760187 27475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22d24bba8b0640e48aab24f27933ded3, State: Initialized, Role: FOLLOWER
I20260809 16:33:40.760350 27475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d24bba8b0640e48aab24f27933ded3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41935 } }
I20260809 16:33:40.760440 27475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:40.760488 27475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:40.760537 27475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:40.761147 27475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d24bba8b0640e48aab24f27933ded3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41935 } }
I20260809 16:33:40.761276 27475 leader_election.cc:302] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22d24bba8b0640e48aab24f27933ded3; no voters:
I20260809 16:33:40.761565 27475 leader_election.cc:288] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:33:40.761626 27485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:40.761850 27485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [term 1 LEADER]: Becoming Leader. State: Replica: 22d24bba8b0640e48aab24f27933ded3, State: Running, Role: LEADER
I20260809 16:33:40.761934 27475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:40.762003 27485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d24bba8b0640e48aab24f27933ded3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41935 } }
I20260809 16:33:40.762419 27488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22d24bba8b0640e48aab24f27933ded3. Latest consensus state: current_term: 1 leader_uuid: "22d24bba8b0640e48aab24f27933ded3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d24bba8b0640e48aab24f27933ded3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41935 } } }
I20260809 16:33:40.762528 27488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:40.762413 27486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22d24bba8b0640e48aab24f27933ded3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22d24bba8b0640e48aab24f27933ded3" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41935 } } }
I20260809 16:33:40.762786 27502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:40.763033 27486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:40.763545 27502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:40.765161 27502 catalog_manager.cc:1347] Generated new cluster ID: 3a2fef3fbca743d586244abf8502da9f
I20260809 16:33:40.765216 27502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:33:40.772294 27502 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:33:40.772897 27502 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:40.780351 27502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3: Generated new TSK 0
I20260809 16:33:40.780588 27502 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:33:40.784955 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:41935
--builtin_ntp_servers=127.30.23.148:43187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:33:40.911191 27523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:40.911432 27523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:40.911456 27523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:40.912726 27523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:40.912773 27523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:40.912801 27523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:33:40.914343 27523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:41935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:40.914654 27523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:40.915022 27523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:40.920666 27532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:40.920632 27530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:40.920821 27523 server_base.cc:1034] running on GCE node
W20260809 16:33:40.920662 27536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:40.921455 27523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:40.921790 27523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:40.922977 27523 hybrid_clock.cc:648] HybridClock initialized: now 1786293220922952 us; error 40 us; skew 500 ppm
I20260809 16:33:40.924489 27523 webserver.cc:460] Webserver started at http://127.30.23.129:39347/ using document root <none> and password file <none>
I20260809 16:33:40.924782 27523 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:40.924834 27523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:40.924999 27523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:40.926350 27523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "0abf976c95cd4cf9b5c826b63c286639"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "c3552675cbfa752d3d4228476aaaa726"
server_key_iv: "d4e7f799bc6d80531a1d54b70ce4fe49"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.926754 27523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "0abf976c95cd4cf9b5c826b63c286639"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "c3552675cbfa752d3d4228476aaaa726"
server_key_iv: "d4e7f799bc6d80531a1d54b70ce4fe49"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.928642 27523 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 16:33:40.929792 27543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:40.930003 27523 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:33:40.930127 27523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "0abf976c95cd4cf9b5c826b63c286639"
format_stamp: "Formatted at 2026-08-09 16:33:40 on dist-test-slave-j2vl"
server_key: "c3552675cbfa752d3d4228476aaaa726"
server_key_iv: "d4e7f799bc6d80531a1d54b70ce4fe49"
server_key_version: "encryptionkey@0"
I20260809 16:33:40.930217 27523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:40.963403 27523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:40.963798 27523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:40.963974 27523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:40.964197 27523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:40.964701 27523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:40.964747 27523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:40.964773 27523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:40.964815 27523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:40.971267 27523 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:33699
I20260809 16:33:40.971367 27717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:33699 every 8 connection(s)
I20260809 16:33:40.971757 27523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:33:40.973162 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 27523
I20260809 16:33:40.973269 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:33:40.973534 30814 external_mini_cluster.cc:1428] Setting key e97f0c5fe1d05f071768026d40808d0c
I20260809 16:33:40.974664 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:41935
--builtin_ntp_servers=127.30.23.148:43187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:33:40.978708 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.972703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:50071 (local address 127.30.23.190:41935)
0809 16:33:40.972816 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:40.972818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:40.973004 (+ 186us) server_negotiation.cc:407] Connection header received
0809 16:33:40.973608 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:40.973610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:40.973626 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:40.973708 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:40.974765 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.975314 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.976768 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.977100 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.977200 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.978343 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:40.978344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:40.978344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:40.978349 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:40.978365 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:40.978369 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:40.978431 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:40.978588 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 16:33:40.978630 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 16:33:40.978952 27722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:40.972604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:41935 (local address 127.30.23.129:50071)
0809 16:33:40.972804 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:40.972848 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:40.973486 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:40.973890 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:40.973894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:40.973905 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:40.974636 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.974643 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.975416 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.975418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.976663 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:40.976666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:40.977239 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:40.977240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:40.977474 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:40.978146 (+ 672us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:33:40.978147 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:40.978191 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:33:40.978224 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:40.978453 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:40.978461 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:40.978465 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:40.978546 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:33:40.978678 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260809 16:33:40.979527 27719 heartbeater.cc:347] Connected to a master server at 127.30.23.190:41935
I20260809 16:33:40.979656 27719 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:40.979902 27719 heartbeater.cc:511] Master 127.30.23.190:41935 requested a full tablet report, sending...
I20260809 16:33:40.980587 27410 ts_manager.cc:194] Registered new tserver with Master: 0abf976c95cd4cf9b5c826b63c286639 (127.30.23.129:33699)
I20260809 16:33:40.981282 27410 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.23.129:50071
W20260809 16:33:41.067451 27724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:41.067741 27724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:41.067770 27724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:41.069793 27724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:41.069868 27724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:41.069903 27724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:33:41.071853 27724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:41935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:41.072105 27724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:41.072398 27724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:41.078692 27737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:41.078692 27736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:41.078857 27742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:41.079728 27724 server_base.cc:1034] running on GCE node
I20260809 16:33:41.080014 27724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:41.080331 27724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:41.081580 27724 hybrid_clock.cc:648] HybridClock initialized: now 1786293221081557 us; error 38 us; skew 500 ppm
I20260809 16:33:41.083062 27724 webserver.cc:460] Webserver started at http://127.30.23.130:37887/ using document root <none> and password file <none>
I20260809 16:33:41.083391 27724 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:41.083451 27724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:41.083645 27724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:41.084933 27724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "97ef0b92a84342e48a80e49fbee3be29"
format_stamp: "Formatted at 2026-08-09 16:33:41 on dist-test-slave-j2vl"
server_key: "2efaa24a01514c57eccded04e81ab390"
server_key_iv: "ce2ec9740c4b68250376ea85f85da21e"
server_key_version: "encryptionkey@0"
I20260809 16:33:41.085433 27724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "97ef0b92a84342e48a80e49fbee3be29"
format_stamp: "Formatted at 2026-08-09 16:33:41 on dist-test-slave-j2vl"
server_key: "2efaa24a01514c57eccded04e81ab390"
server_key_iv: "ce2ec9740c4b68250376ea85f85da21e"
server_key_version: "encryptionkey@0"
I20260809 16:33:41.087296 27724 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:33:41.088281 27751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:41.088482 27724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:41.088579 27724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "97ef0b92a84342e48a80e49fbee3be29"
format_stamp: "Formatted at 2026-08-09 16:33:41 on dist-test-slave-j2vl"
server_key: "2efaa24a01514c57eccded04e81ab390"
server_key_iv: "ce2ec9740c4b68250376ea85f85da21e"
server_key_version: "encryptionkey@0"
I20260809 16:33:41.088642 27724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:41.103353 27724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:41.103677 27724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:41.103788 27724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:41.103992 27724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:41.104399 27724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:41.104439 27724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:41.104462 27724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:41.104475 27724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:41.111239 27724 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:46847
I20260809 16:33:41.111306 27936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:46847 every 8 connection(s)
I20260809 16:33:41.111678 27724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:33:41.112007 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 27724
I20260809 16:33:41.112131 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:33:41.112391 30814 external_mini_cluster.cc:1428] Setting key 04d088602b7b667dc6e7c72ec23099ba
I20260809 16:33:41.113689 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:41935
--builtin_ntp_servers=127.30.23.148:43187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:33:41.118670 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.112691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:52437 (local address 127.30.23.190:41935)
0809 16:33:41.112777 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:41.112779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:41.113089 (+ 310us) server_negotiation.cc:407] Connection header received
0809 16:33:41.113724 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:41.113725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:41.113738 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:41.113790 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:41.114528 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.115118 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.116677 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.117000 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.117077 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.118262 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:41.118264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:41.118265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:41.118271 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:41.118289 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:41.118294 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:41.118365 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:41.118503 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 16:33:41.118537 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 16:33:41.118801 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.112651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:41935 (local address 127.30.23.130:52437)
0809 16:33:41.112892 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:41.112939 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:41.113612 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:41.113877 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:41.113881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:41.113889 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:41.114428 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.114432 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.115234 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.115235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.116541 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.116545 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.117231 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.117233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.117446 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.118098 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:33:41.118099 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:41.118131 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:33:41.118158 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:41.118396 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:41.118402 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:41.118414 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 16:33:41.118476 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:33:41.118575 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":105,"threads_started":1}
I20260809 16:33:41.119591 27938 heartbeater.cc:347] Connected to a master server at 127.30.23.190:41935
I20260809 16:33:41.119700 27938 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:41.119953 27938 heartbeater.cc:511] Master 127.30.23.190:41935 requested a full tablet report, sending...
I20260809 16:33:41.120484 27410 ts_manager.cc:194] Registered new tserver with Master: 97ef0b92a84342e48a80e49fbee3be29 (127.30.23.130:46847)
I20260809 16:33:41.120905 27410 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.23.130:52437
W20260809 16:33:41.202762 27943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:41.203056 27943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:41.203100 27943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:41.204355 27943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:41.204430 27943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:41.204489 27943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:33:41.205981 27943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:41935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:41.206290 27943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:41.206631 27943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:41.212121 27960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:41.212141 27962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:41.212379 27943 server_base.cc:1034] running on GCE node
W20260809 16:33:41.212141 27959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:41.212738 27943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:41.213011 27943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:41.214202 27943 hybrid_clock.cc:648] HybridClock initialized: now 1786293221214209 us; error 68 us; skew 500 ppm
I20260809 16:33:41.215656 27943 webserver.cc:460] Webserver started at http://127.30.23.131:46067/ using document root <none> and password file <none>
I20260809 16:33:41.215938 27943 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:41.216017 27943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:41.216174 27943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:41.217537 27943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "fe1e1eb92996446cb49f0a934054dc79"
format_stamp: "Formatted at 2026-08-09 16:33:41 on dist-test-slave-j2vl"
server_key: "449d741cc43fed27ab52ad08921d3971"
server_key_iv: "4f7ec293c552f4a4106675724efc06e8"
server_key_version: "encryptionkey@0"
I20260809 16:33:41.217926 27943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "fe1e1eb92996446cb49f0a934054dc79"
format_stamp: "Formatted at 2026-08-09 16:33:41 on dist-test-slave-j2vl"
server_key: "449d741cc43fed27ab52ad08921d3971"
server_key_iv: "4f7ec293c552f4a4106675724efc06e8"
server_key_version: "encryptionkey@0"
I20260809 16:33:41.219542 27943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:33:41.220444 27971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:41.220679 27943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:33:41.220813 27943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "fe1e1eb92996446cb49f0a934054dc79"
format_stamp: "Formatted at 2026-08-09 16:33:41 on dist-test-slave-j2vl"
server_key: "449d741cc43fed27ab52ad08921d3971"
server_key_iv: "4f7ec293c552f4a4106675724efc06e8"
server_key_version: "encryptionkey@0"
I20260809 16:33:41.220901 27943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:41.236773 27943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:41.237180 27943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:41.237521 27943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:41.237833 27943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:41.238165 27943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:41.238202 27943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:41.238227 27943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:41.238240 27943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:41.244719 27943 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:40311
I20260809 16:33:41.244757 28152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:40311 every 8 connection(s)
I20260809 16:33:41.245244 27943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:33:41.251533 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 27943
I20260809 16:33:41.251690 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:33:41.251791 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.246224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:52095 (local address 127.30.23.190:41935)
0809 16:33:41.246341 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:41.246343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:41.246493 (+ 150us) server_negotiation.cc:407] Connection header received
0809 16:33:41.247048 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:41.247050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:41.247063 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:41.247137 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:41.247977 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.248481 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.249837 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.250035 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.250107 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.251331 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:41.251332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:41.251332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:41.251339 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:41.251354 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:41.251359 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:41.251409 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:41.251617 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 16:33:41.251684 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 16:33:41.251947 30814 external_mini_cluster.cc:1428] Setting key 6eb75e36ee15c70d81788722b837135b
I20260809 16:33:41.251982 28157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.246134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:41935 (local address 127.30.23.131:52095)
0809 16:33:41.246354 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:41.246398 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:41.246905 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:41.247232 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:41.247236 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:41.247246 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:41.247784 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.247788 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.248596 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.248597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.249687 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.249700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.250156 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.250157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.250292 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.251115 (+ 823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:33:41.251116 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:41.251150 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:33:41.251180 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:41.251437 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:41.251446 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:41.251450 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:41.251545 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 16:33:41.251695 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260809 16:33:41.252540 28154 heartbeater.cc:347] Connected to a master server at 127.30.23.190:41935
I20260809 16:33:41.252671 28154 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:41.252880 28154 heartbeater.cc:511] Master 127.30.23.190:41935 requested a full tablet report, sending...
I20260809 16:33:41.253499 27409 ts_manager.cc:194] Registered new tserver with Master: fe1e1eb92996446cb49f0a934054dc79 (127.30.23.131:40311)
I20260809 16:33:41.253890 27409 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.23.131:52095
I20260809 16:33:41.264204 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:33:41.269133 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.265755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50024 (local address 127.30.23.190:41935)
0809 16:33:41.265882 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:41.265885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:41.265899 (+ 14us) server_negotiation.cc:407] Connection header received
0809 16:33:41.265977 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:41.265977 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:41.265988 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:41.266048 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:41.266258 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.266817 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.267675 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.267910 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.267976 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.268755 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:41.268756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:41.268757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:41.268762 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:41.268777 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:41.268800 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:41.268852 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:41.268963 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 16:33:41.268989 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 16:33:41.271306 27409 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:33:41.273193 27409 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:33:41.280606 28174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.276209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:40311 (local address 127.0.0.1:38600)
0809 16:33:41.276604 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:41.276616 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:41.276678 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:41.276963 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:41.276964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:41.276967 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:41.277032 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.277034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.278415 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.278416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.279500 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.279502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.280220 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.280222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.280314 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.280329 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:33:41.280435 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 16:33:41.280502 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":40,"threads_started":1}
I20260809 16:33:41.280673 28169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.276119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48966 (local address 127.30.23.130:46847)
0809 16:33:41.276681 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:41.276685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:41.276696 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:33:41.276750 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:41.276753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:41.276771 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:41.277031 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:41.277091 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.278171 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.279477 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.280269 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.280384 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.280464 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:41.280540 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 16:33:41.280584 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":90,"threads_started":1}
I20260809 16:33:41.280611 28164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.275968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:46847 (local address 127.0.0.1:48966)
0809 16:33:41.276275 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:41.276318 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:41.276395 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:41.276894 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:41.276896 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:41.276898 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:41.276951 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.276953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.278254 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.278256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.279383 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.279386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.280412 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.280413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.280472 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.280475 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:33:41.280526 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:33:41.280558 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":73,"threads_started":1}
I20260809 16:33:41.281054 28176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.276394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38600 (local address 127.30.23.131:40311)
0809 16:33:41.276606 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:41.276611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:41.276665 (+ 54us) server_negotiation.cc:407] Connection header received
0809 16:33:41.276779 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:41.276782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:41.276810 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:41.276891 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:41.277097 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.277990 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.279594 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.280123 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.280222 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.280328 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:41.280894 (+ 566us) server_negotiation.cc:299] Negotiation successful
0809 16:33:41.280954 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":119,"threads_started":1}
I20260809 16:33:41.281068 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.275887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:33699 (local address 127.0.0.1:53150)
0809 16:33:41.276197 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:41.276246 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:41.276329 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:41.276554 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:41.276556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:41.276558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:41.276634 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.276636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.278081 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.278082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.279452 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.279456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.280483 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.280484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.280552 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.280556 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:41.280606 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:33:41.280652 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":66,"thread_start_us":225,"threads_started":1}
I20260809 16:33:41.281426 28167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.276044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53150 (local address 127.30.23.129:33699)
0809 16:33:41.276350 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:41.276355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:41.276369 (+ 14us) server_negotiation.cc:407] Connection header received
0809 16:33:41.276427 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:41.276429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:41.276452 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:41.276534 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:41.276725 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.277983 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.279536 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.280276 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.280349 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.280444 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:41.280737 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 16:33:41.281073 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":125,"threads_started":1}
I20260809 16:33:41.281841 28044 tablet_service.cc:1467] Processing CreateTablet for tablet e54fe93c036d466da1c6602b23fc56f4 (DEFAULT_TABLE table=test-table [id=b3c186ff6ea145e2a598e92bdd3a6050]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:41.281837 27825 tablet_service.cc:1467] Processing CreateTablet for tablet e54fe93c036d466da1c6602b23fc56f4 (DEFAULT_TABLE table=test-table [id=b3c186ff6ea145e2a598e92bdd3a6050]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:41.282033 27623 tablet_service.cc:1467] Processing CreateTablet for tablet e54fe93c036d466da1c6602b23fc56f4 (DEFAULT_TABLE table=test-table [id=b3c186ff6ea145e2a598e92bdd3a6050]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:41.282173 28044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54fe93c036d466da1c6602b23fc56f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:41.282219 27825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54fe93c036d466da1c6602b23fc56f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:41.282331 27623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54fe93c036d466da1c6602b23fc56f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:41.284885 28177 tablet_bootstrap.cc:492] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79: Bootstrap starting.
I20260809 16:33:41.285688 28177 tablet_bootstrap.cc:654] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:41.285805 28181 tablet_bootstrap.cc:492] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639: Bootstrap starting.
I20260809 16:33:41.286038 28177 log.cc:824] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:41.286481 28181 tablet_bootstrap.cc:654] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:41.286727 28177 tablet_bootstrap.cc:492] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79: No bootstrap required, opened a new log
I20260809 16:33:41.286787 28177 ts_tablet_manager.cc:1397] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:33:41.286818 28181 log.cc:824] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:41.287153 28178 tablet_bootstrap.cc:492] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29: Bootstrap starting.
I20260809 16:33:41.287817 28181 tablet_bootstrap.cc:492] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639: No bootstrap required, opened a new log
I20260809 16:33:41.287874 28178 tablet_bootstrap.cc:654] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:41.287984 28181 ts_tablet_manager.cc:1397] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:33:41.288213 28178 log.cc:824] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:41.289127 28178 tablet_bootstrap.cc:492] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29: No bootstrap required, opened a new log
I20260809 16:33:41.289206 28178 ts_tablet_manager.cc:1397] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:33:41.289594 28177 raft_consensus.cc:348] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ef0b92a84342e48a80e49fbee3be29" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46847 } } peers { permanent_uuid: "0abf976c95cd4cf9b5c826b63c286639" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33699 } } peers { permanent_uuid: "fe1e1eb92996446cb49f0a934054dc79" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40311 } }
I20260809 16:33:41.289898 28177 raft_consensus.cc:374] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:41.289959 28177 raft_consensus.cc:729] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe1e1eb92996446cb49f0a934054dc79, State: Initialized, Role: FOLLOWER
I20260809 16:33:41.290091 28177 consensus_queue.cc:260] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ef0b92a84342e48a80e49fbee3be29" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46847 } } peers { permanent_uuid: "0abf976c95cd4cf9b5c826b63c286639" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33699 } } peers { permanent_uuid: "fe1e1eb92996446cb49f0a934054dc79" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40311 } }
I20260809 16:33:41.290474 28154 heartbeater.cc:503] Master 127.30.23.190:41935 was elected leader, sending a full tablet report...
I20260809 16:33:41.290462 28177 ts_tablet_manager.cc:1428] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 16:33:41.291108 28181 raft_consensus.cc:348] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ef0b92a84342e48a80e49fbee3be29" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46847 } } peers { permanent_uuid: "0abf976c95cd4cf9b5c826b63c286639" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33699 } } peers { permanent_uuid: "fe1e1eb92996446cb49f0a934054dc79" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40311 } }
I20260809 16:33:41.291291 28181 raft_consensus.cc:374] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:41.291328 28181 raft_consensus.cc:729] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0abf976c95cd4cf9b5c826b63c286639, State: Initialized, Role: FOLLOWER
I20260809 16:33:41.291440 28181 consensus_queue.cc:260] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ef0b92a84342e48a80e49fbee3be29" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46847 } } peers { permanent_uuid: "0abf976c95cd4cf9b5c826b63c286639" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33699 } } peers { permanent_uuid: "fe1e1eb92996446cb49f0a934054dc79" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40311 } }
I20260809 16:33:41.291705 28181 ts_tablet_manager.cc:1428] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 16:33:41.291879 27719 heartbeater.cc:503] Master 127.30.23.190:41935 was elected leader, sending a full tablet report...
I20260809 16:33:41.292542 28178 raft_consensus.cc:348] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ef0b92a84342e48a80e49fbee3be29" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46847 } } peers { permanent_uuid: "0abf976c95cd4cf9b5c826b63c286639" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33699 } } peers { permanent_uuid: "fe1e1eb92996446cb49f0a934054dc79" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40311 } }
I20260809 16:33:41.292701 28178 raft_consensus.cc:374] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:41.292723 28178 raft_consensus.cc:729] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97ef0b92a84342e48a80e49fbee3be29, State: Initialized, Role: FOLLOWER
I20260809 16:33:41.292860 28178 consensus_queue.cc:260] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ef0b92a84342e48a80e49fbee3be29" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46847 } } peers { permanent_uuid: "0abf976c95cd4cf9b5c826b63c286639" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33699 } } peers { permanent_uuid: "fe1e1eb92996446cb49f0a934054dc79" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40311 } }
I20260809 16:33:41.293164 28178 ts_tablet_manager.cc:1428] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 16:33:41.293228 27938 heartbeater.cc:503] Master 127.30.23.190:41935 was elected leader, sending a full tablet report...
W20260809 16:33:41.362397 27939 tablet.cc:2369] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:33:41.392381 28192 raft_consensus.cc:482] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:33:41.392561 28192 raft_consensus.cc:504] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ef0b92a84342e48a80e49fbee3be29" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46847 } } peers { permanent_uuid: "0abf976c95cd4cf9b5c826b63c286639" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33699 } } peers { permanent_uuid: "fe1e1eb92996446cb49f0a934054dc79" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40311 } }
I20260809 16:33:41.392949 28192 leader_election.cc:288] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0abf976c95cd4cf9b5c826b63c286639 (127.30.23.129:33699), fe1e1eb92996446cb49f0a934054dc79 (127.30.23.131:40311)
I20260809 16:33:41.397120 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.393179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:33699 (local address 127.30.23.130:44751)
0809 16:33:41.393244 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:41.393257 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:41.393304 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:41.393525 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:41.393526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:41.393528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:41.393615 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.393617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.394630 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.394632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.395991 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.395996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.396808 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.396809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.396905 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.396912 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:33:41.396986 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:33:41.397043 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":67}
I20260809 16:33:41.397153 28176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.393317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:33597 (local address 127.30.23.131:40311)
0809 16:33:41.393390 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:41.393392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:41.393542 (+ 150us) server_negotiation.cc:407] Connection header received
0809 16:33:41.393580 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:41.393581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:41.393590 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:41.393648 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:41.393796 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.394524 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.396104 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.396691 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.396777 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.396784 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:41.397037 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 16:33:41.397087 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 16:33:41.397147 28167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.393262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:44751 (local address 127.30.23.129:33699)
0809 16:33:41.393369 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:41.393371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:41.393383 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:33:41.393420 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:41.393421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:41.393433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:41.393502 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:41.393724 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.394525 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.396104 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.396690 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.396774 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.396783 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:41.397008 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 16:33:41.397059 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:33:41.397377 28193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.393219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:40311 (local address 127.30.23.130:33597)
0809 16:33:41.393445 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:41.393453 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:41.393524 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:41.393665 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:41.393666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:41.393667 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:41.393728 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.393729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.394630 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.394632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.395991 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:41.395996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.396808 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.396809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:41.396905 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.396912 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:33:41.397226 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 16:33:41.397269 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":63,"thread_start_us":178,"threads_started":1}
I20260809 16:33:41.397568 27650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54fe93c036d466da1c6602b23fc56f4" candidate_uuid: "97ef0b92a84342e48a80e49fbee3be29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0abf976c95cd4cf9b5c826b63c286639" is_pre_election: true
I20260809 16:33:41.397753 27650 raft_consensus.cc:2454] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97ef0b92a84342e48a80e49fbee3be29 in term 0.
I20260809 16:33:41.397716 28078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54fe93c036d466da1c6602b23fc56f4" candidate_uuid: "97ef0b92a84342e48a80e49fbee3be29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe1e1eb92996446cb49f0a934054dc79" is_pre_election: true
I20260809 16:33:41.397850 28078 raft_consensus.cc:2454] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97ef0b92a84342e48a80e49fbee3be29 in term 0.
I20260809 16:33:41.398108 27755 leader_election.cc:302] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97ef0b92a84342e48a80e49fbee3be29, fe1e1eb92996446cb49f0a934054dc79; no voters:
I20260809 16:33:41.398278 28192 raft_consensus.cc:2781] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:33:41.398350 28192 raft_consensus.cc:482] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:33:41.398368 28192 raft_consensus.cc:3037] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:41.399145 28192 raft_consensus.cc:504] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ef0b92a84342e48a80e49fbee3be29" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46847 } } peers { permanent_uuid: "0abf976c95cd4cf9b5c826b63c286639" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33699 } } peers { permanent_uuid: "fe1e1eb92996446cb49f0a934054dc79" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40311 } }
I20260809 16:33:41.399298 28192 leader_election.cc:288] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [CANDIDATE]: Term 1 election: Requested vote from peers 0abf976c95cd4cf9b5c826b63c286639 (127.30.23.129:33699), fe1e1eb92996446cb49f0a934054dc79 (127.30.23.131:40311)
I20260809 16:33:41.399483 28078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54fe93c036d466da1c6602b23fc56f4" candidate_uuid: "97ef0b92a84342e48a80e49fbee3be29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe1e1eb92996446cb49f0a934054dc79"
I20260809 16:33:41.399573 28078 raft_consensus.cc:3037] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:41.400247 28078 raft_consensus.cc:2454] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97ef0b92a84342e48a80e49fbee3be29 in term 1.
I20260809 16:33:41.400388 27650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54fe93c036d466da1c6602b23fc56f4" candidate_uuid: "97ef0b92a84342e48a80e49fbee3be29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0abf976c95cd4cf9b5c826b63c286639"
I20260809 16:33:41.400460 27755 leader_election.cc:302] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97ef0b92a84342e48a80e49fbee3be29, fe1e1eb92996446cb49f0a934054dc79; no voters:
I20260809 16:33:41.400542 28192 raft_consensus.cc:2781] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:41.400486 27650 raft_consensus.cc:3037] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:41.400722 28192 raft_consensus.cc:686] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [term 1 LEADER]: Becoming Leader. State: Replica: 97ef0b92a84342e48a80e49fbee3be29, State: Running, Role: LEADER
I20260809 16:33:41.400941 28192 consensus_queue.cc:237] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ef0b92a84342e48a80e49fbee3be29" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46847 } } peers { permanent_uuid: "0abf976c95cd4cf9b5c826b63c286639" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33699 } } peers { permanent_uuid: "fe1e1eb92996446cb49f0a934054dc79" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40311 } }
I20260809 16:33:41.401530 27650 raft_consensus.cc:2454] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97ef0b92a84342e48a80e49fbee3be29 in term 1.
I20260809 16:33:41.401945 27409 catalog_manager.cc:5495] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97ef0b92a84342e48a80e49fbee3be29 (127.30.23.130). New cstate: current_term: 1 leader_uuid: "97ef0b92a84342e48a80e49fbee3be29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97ef0b92a84342e48a80e49fbee3be29" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 46847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0abf976c95cd4cf9b5c826b63c286639" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe1e1eb92996446cb49f0a934054dc79" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40311 } health_report { overall_health: UNKNOWN } } }
I20260809 16:33:41.431671 28169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.428929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48980 (local address 127.30.23.130:46847)
0809 16:33:41.428997 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:41.429000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:41.429009 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:33:41.429049 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:41.429050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:41.429070 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:41.429143 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:33:41.429269 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.429771 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.430601 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:41.430795 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:41.430869 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:41.431195 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:33:41.431320 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:33:41.431407 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:41.431564 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 16:33:41.431590 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:33:41.433799 27650 raft_consensus.cc:1261] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639 [term 1 FOLLOWER]: Refusing update from remote peer 97ef0b92a84342e48a80e49fbee3be29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:41.433779 28078 raft_consensus.cc:1261] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79 [term 1 FOLLOWER]: Refusing update from remote peer 97ef0b92a84342e48a80e49fbee3be29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:41.434201 28194 consensus_queue.cc:1035] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0abf976c95cd4cf9b5c826b63c286639" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:41.434444 28192 consensus_queue.cc:1035] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe1e1eb92996446cb49f0a934054dc79" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 40311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:41.436654 28197 mvcc.cc:205] Tried to move back new op lower bound from 7316657034989318144 to 7316657034859003904. Current Snapshot: MvccSnapshot[applied={T|T < 7316657034989318144}]
I20260809 16:33:41.437402 28199 mvcc.cc:205] Tried to move back new op lower bound from 7316657034989318144 to 7316657034859003904. Current Snapshot: MvccSnapshot[applied={T|T < 7316657034989318144}]
I20260809 16:33:41.437932 28198 mvcc.cc:205] Tried to move back new op lower bound from 7316657034989318144 to 7316657034859003904. Current Snapshot: MvccSnapshot[applied={T|T < 7316657034989318144 or (T in {7316657034989318144})}]
I20260809 16:33:41.439461 27410 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:41.439555 27410 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:41.440516 27410 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22d24bba8b0640e48aab24f27933ded3: Sending DeleteTablet for 3 replicas of tablet e54fe93c036d466da1c6602b23fc56f4
I20260809 16:33:41.440974 27825 tablet_service.cc:1514] Processing DeleteTablet for tablet e54fe93c036d466da1c6602b23fc56f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:41 UTC) from {username='slave'} at 127.0.0.1:48966
I20260809 16:33:41.440996 27623 tablet_service.cc:1514] Processing DeleteTablet for tablet e54fe93c036d466da1c6602b23fc56f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:41 UTC) from {username='slave'} at 127.0.0.1:53150
I20260809 16:33:41.441228 28044 tablet_service.cc:1514] Processing DeleteTablet for tablet e54fe93c036d466da1c6602b23fc56f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:41 UTC) from {username='slave'} at 127.0.0.1:38600
I20260809 16:33:41.441258 28214 tablet_replica.cc:330] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29: stopping tablet replica
I20260809 16:33:41.441412 28214 raft_consensus.cc:2229] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:33:41.441557 28214 raft_consensus.cc:2258] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:41.441730 28213 tablet_replica.cc:330] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639: stopping tablet replica
I20260809 16:33:41.441851 28213 raft_consensus.cc:2229] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:41.441965 28213 raft_consensus.cc:2258] T e54fe93c036d466da1c6602b23fc56f4 P 0abf976c95cd4cf9b5c826b63c286639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:41.441974 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 27523
I20260809 16:33:41.442611 28214 ts_tablet_manager.cc:1905] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:41.445070 28214 ts_tablet_manager.cc:1918] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:41.445240 28214 log.cc:1196] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/e54fe93c036d466da1c6602b23fc56f4
I20260809 16:33:41.445624 28214 ts_tablet_manager.cc:1939] T e54fe93c036d466da1c6602b23fc56f4 P 97ef0b92a84342e48a80e49fbee3be29: Deleting consensus metadata
I20260809 16:33:41.446215 27389 catalog_manager.cc:4841] TS 97ef0b92a84342e48a80e49fbee3be29 (127.30.23.130:46847): tablet e54fe93c036d466da1c6602b23fc56f4 (table test-table [id=b3c186ff6ea145e2a598e92bdd3a6050]) successfully deleted
I20260809 16:33:41.447125 28216 tablet_replica.cc:330] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79: stopping tablet replica
I20260809 16:33:41.447291 28216 raft_consensus.cc:2229] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:41.447420 28216 raft_consensus.cc:2258] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:41.447897 28216 ts_tablet_manager.cc:1905] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:41.449707 28216 ts_tablet_manager.cc:1918] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:41.449795 28216 log.cc:1196] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/e54fe93c036d466da1c6602b23fc56f4
I20260809 16:33:41.450067 28216 ts_tablet_manager.cc:1939] T e54fe93c036d466da1c6602b23fc56f4 P fe1e1eb92996446cb49f0a934054dc79: Deleting consensus metadata
I20260809 16:33:41.450647 27389 catalog_manager.cc:4841] TS fe1e1eb92996446cb49f0a934054dc79 (127.30.23.131:40311): tablet e54fe93c036d466da1c6602b23fc56f4 (table test-table [id=b3c186ff6ea145e2a598e92bdd3a6050]) successfully deleted
W20260809 16:33:41.457679 27388 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:33699 (error 108)
I20260809 16:33:41.457886 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 27724
I20260809 16:33:41.465580 28174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:41.458166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:33699 (local address 127.0.0.1:53152)
0809 16:33:41.465422 (+ 7256us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:41.465518 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7141}
W20260809 16:33:41.469934 27388 catalog_manager.cc:4584] TS 0abf976c95cd4cf9b5c826b63c286639 (127.30.23.129:33699): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e54fe93c036d466da1c6602b23fc56f4: Network error: Client connection negotiation failed: client connection to 127.30.23.129:33699: connect: Connection refused (error 111)
I20260809 16:33:41.474301 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 27943
I20260809 16:33:41.489921 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 27364
2026-08-09T16:33:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (912 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:33:41 dist-test-slave-j2vl krb5kdc[28223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:33:41 dist-test-slave-j2vl krb5kdc[28223](info): set up 2 sockets
Aug 09 16:33:41 dist-test-slave-j2vl krb5kdc[28223](info): commencing operation
krb5kdc: starting...
W20260809 16:33:42.777256 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:33:42 dist-test-slave-j2vl krb5kdc[28223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:33:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:33:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:33:42.921522 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:33261
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:44523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:33261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:43.010056 28246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:43.010298 28246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:43.010337 28246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:43.011536 28246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:43.011605 28246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:43.011632 28246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:43.011664 28246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:43.011691 28246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:43.013232 28246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:44523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:33261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:33261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:43.013548 28246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:43.013796 28246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:43.019042 28256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:43.019080 28254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:43.019075 28246 server_base.cc:1034] running on GCE node
W20260809 16:33:43.019055 28258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:43.019474 28246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:43.019783 28246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:43.020977 28246 hybrid_clock.cc:648] HybridClock initialized: now 1786293223020949 us; error 44 us; skew 500 ppm
Aug 09 16:33:43 dist-test-slave-j2vl krb5kdc[28223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293223, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:43.023741 28246 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:33:43.024268 28246 webserver.cc:460] Webserver started at http://127.30.23.190:41737/ using document root <none> and password file <none>
I20260809 16:33:43.024509 28246 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:43.024554 28246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:43.024704 28246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:43.025866 28246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "d3dd41f572e54038a5361a3ab70216ba"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "d59ec57bc911107e9b47ceb6ae332a3e"
server_key_iv: "12d31af79b5fbf8ce188cc0a86c7142b"
server_key_version: "encryptionkey@0"
I20260809 16:33:43.026230 28246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "d3dd41f572e54038a5361a3ab70216ba"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "d59ec57bc911107e9b47ceb6ae332a3e"
server_key_iv: "12d31af79b5fbf8ce188cc0a86c7142b"
server_key_version: "encryptionkey@0"
I20260809 16:33:43.027727 28246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:33:43.028635 28265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:43.028918 28246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:43.029047 28246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "d3dd41f572e54038a5361a3ab70216ba"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "d59ec57bc911107e9b47ceb6ae332a3e"
server_key_iv: "12d31af79b5fbf8ce188cc0a86c7142b"
server_key_version: "encryptionkey@0"
I20260809 16:33:43.029141 28246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:43.038717 28246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:43.041028 28246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:43.041218 28246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:43.045816 28246 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:33261
I20260809 16:33:43.045866 28349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:33261 every 8 connection(s)
I20260809 16:33:43.046271 28246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:43.047289 28351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:43.048064 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 28246
I20260809 16:33:43.048161 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:33:43.048400 30814 external_mini_cluster.cc:1428] Setting key ffb4ef51e33b3a54b16de49c84190014
I20260809 16:33:43.050853 28351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba: Bootstrap starting.
I20260809 16:33:43.051479 28351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:43.051810 28351 log.cc:824] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:43.052631 28351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba: No bootstrap required, opened a new log
I20260809 16:33:43.055431 28351 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dd41f572e54038a5361a3ab70216ba" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33261 } }
I20260809 16:33:43.055593 28351 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:43.055615 28351 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3dd41f572e54038a5361a3ab70216ba, State: Initialized, Role: FOLLOWER
I20260809 16:33:43.055752 28351 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dd41f572e54038a5361a3ab70216ba" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33261 } }
I20260809 16:33:43.055841 28351 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:43.055866 28351 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:43.055934 28351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [term 0 FOLLOWER]: Advancing to term 1
Aug 09 16:33:43 dist-test-slave-j2vl krb5kdc[28223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:33:43.056604 28351 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dd41f572e54038a5361a3ab70216ba" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33261 } }
I20260809 16:33:43.056744 28351 leader_election.cc:302] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3dd41f572e54038a5361a3ab70216ba; no voters:
I20260809 16:33:43.056972 28351 leader_election.cc:288] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:33:43.057009 28356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:43.057132 28356 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [term 1 LEADER]: Becoming Leader. State: Replica: d3dd41f572e54038a5361a3ab70216ba, State: Running, Role: LEADER
I20260809 16:33:43.057287 28351 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:43.057309 28356 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dd41f572e54038a5361a3ab70216ba" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33261 } }
I20260809 16:33:43.057881 28358 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3dd41f572e54038a5361a3ab70216ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dd41f572e54038a5361a3ab70216ba" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33261 } } }
I20260809 16:33:43.057904 28362 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3dd41f572e54038a5361a3ab70216ba. Latest consensus state: current_term: 1 leader_uuid: "d3dd41f572e54038a5361a3ab70216ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dd41f572e54038a5361a3ab70216ba" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 33261 } } }
I20260809 16:33:43.058069 28358 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:43.058082 28362 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:43.058815 28373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:43.059401 28373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:43.061673 28354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.049033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.30.23.190:33261)
0809 16:33:43.049212 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:43.049240 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:43.049268 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:33:43.049795 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:43.049803 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:43.049855 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:43.049974 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:43.050827 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.051889 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.052988 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.053494 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.053585 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.056936 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:43.056946 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:43.056946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:43.056958 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:43.059591 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:43.060056 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:43.060060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:43.060061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:43.060181 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:43.060436 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:43.060437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:43.060437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:43.060798 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:43.061008 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:43.061258 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 16:33:43.061456 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260809 16:33:43.061725 28373 catalog_manager.cc:1347] Generated new cluster ID: b96e640a3660408bb763b28d41ea9a0c
I20260809 16:33:43.061774 28373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:33:43.076521 28373 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:33:43.077118 28373 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:43.085489 28373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba: Generated new TSK 0
I20260809 16:33:43.085711 28373 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:33:43.121776 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33261
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:44523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:43.221885 28390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:43.222118 28390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:43.222160 28390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:43.223373 28390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:43.223440 28390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:43.223486 28390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:33:43.225097 28390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:44523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:43.225435 28390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:43.225692 28390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:43.231184 28398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:43.231180 28396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:43.231213 28390 server_base.cc:1034] running on GCE node
W20260809 16:33:43.231155 28400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:43.231732 28390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:43.232046 28390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:43.233222 28390 hybrid_clock.cc:648] HybridClock initialized: now 1786293223233207 us; error 32 us; skew 500 ppm
Aug 09 16:33:43 dist-test-slave-j2vl krb5kdc[28223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293223, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:43.236371 28390 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:33:43.236910 28390 webserver.cc:460] Webserver started at http://127.30.23.129:37445/ using document root <none> and password file <none>
I20260809 16:33:43.237190 28390 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:43.237263 28390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:43.237445 28390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:43.238561 28390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "7e360ae130a44121a891e5da0a369455"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "e050795819743f7f5f074cef283571a7"
server_key_iv: "f4843fdb14e7129ac640d1371f9f8c85"
server_key_version: "encryptionkey@0"
I20260809 16:33:43.238948 28390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "7e360ae130a44121a891e5da0a369455"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "e050795819743f7f5f074cef283571a7"
server_key_iv: "f4843fdb14e7129ac640d1371f9f8c85"
server_key_version: "encryptionkey@0"
I20260809 16:33:43.240381 28390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:33:43.241256 28409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:43.241483 28390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:43.241605 28390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "7e360ae130a44121a891e5da0a369455"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "e050795819743f7f5f074cef283571a7"
server_key_iv: "f4843fdb14e7129ac640d1371f9f8c85"
server_key_version: "encryptionkey@0"
I20260809 16:33:43.241689 28390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:43.250237 28390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:43.252612 28390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:43.252786 28390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:43.253057 28390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:43.253463 28390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:43.253531 28390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:43.253569 28390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:43.253610 28390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:43.259929 28390 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:46739
I20260809 16:33:43.260031 28594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:46739 every 8 connection(s)
I20260809 16:33:43.260351 28390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:33:43 dist-test-slave-j2vl krb5kdc[28223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293223, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:33:43.269065 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 28390
I20260809 16:33:43.269198 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:33:43.269479 30814 external_mini_cluster.cc:1428] Setting key ca7a5372335e1555752d66c5021f5b8d
I20260809 16:33:43.270828 28354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.261168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:51699 (local address 127.30.23.190:33261)
0809 16:33:43.261243 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:43.261244 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:43.261504 (+ 260us) server_negotiation.cc:407] Connection header received
0809 16:33:43.261978 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:43.261979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:43.261991 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:43.262061 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:43.263124 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.263693 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.264724 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.264957 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.265024 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.267572 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:43.267574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:43.267574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:43.267581 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:43.268978 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:43.269603 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:43.269605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:43.269605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:43.269685 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:43.270033 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:43.270034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:43.270034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:43.270199 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:43.270265 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:43.270614 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 16:33:43.270716 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 16:33:43.270859 28598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.261137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33261 (local address 127.30.23.129:51699)
0809 16:33:43.261320 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:43.261383 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:43.261856 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:43.262159 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:43.262163 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:43.262810 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:43.263011 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.263016 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.263799 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.263800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.264627 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.264630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.265060 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.265062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.265176 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.265816 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:43.265825 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:43.267446 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:43.269077 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:43.269093 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:43.269095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:43.269493 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:43.269781 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:43.269782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:43.269782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:43.269948 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:43.270297 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:43.270305 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:43.270458 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 16:33:43.270546 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:33:43.270678 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260809 16:33:43.271440 28596 heartbeater.cc:347] Connected to a master server at 127.30.23.190:33261
I20260809 16:33:43.271559 28596 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:43.271744 28596 heartbeater.cc:511] Master 127.30.23.190:33261 requested a full tablet report, sending...
I20260809 16:33:43.272344 28291 ts_manager.cc:194] Registered new tserver with Master: 7e360ae130a44121a891e5da0a369455 (127.30.23.129:46739)
I20260809 16:33:43.273002 28291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:51699
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:33:43.302464 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33261
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:44523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:43.398272 28601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:43.398715 28601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:43.398818 28601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:43.401197 28601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:43.401283 28601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:43.401332 28601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:33:43.403641 28601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:44523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:43.403939 28601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:43.404224 28601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:43.411528 28613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:43.411657 28610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:43.411538 28609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:43.411953 28601 server_base.cc:1034] running on GCE node
I20260809 16:33:43.412218 28601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:43.412640 28601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:43.413823 28601 hybrid_clock.cc:648] HybridClock initialized: now 1786293223413795 us; error 50 us; skew 500 ppm
Aug 09 16:33:43 dist-test-slave-j2vl krb5kdc[28223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293223, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:43.417310 28601 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:33:43.417898 28601 webserver.cc:460] Webserver started at http://127.30.23.130:33137/ using document root <none> and password file <none>
I20260809 16:33:43.418211 28601 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:43.418283 28601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:43.418438 28601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:43.419797 28601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "af99a58898384ed98dd89a527f4af713"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "8bbb2bdd492213faa88dd5c1d6560fc7"
server_key_iv: "f4f78b41c596eac2c9a6841391e71026"
server_key_version: "encryptionkey@0"
I20260809 16:33:43.420231 28601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "af99a58898384ed98dd89a527f4af713"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "8bbb2bdd492213faa88dd5c1d6560fc7"
server_key_iv: "f4f78b41c596eac2c9a6841391e71026"
server_key_version: "encryptionkey@0"
I20260809 16:33:43.422317 28601 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 16:33:43.423238 28626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:43.423442 28601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:43.423542 28601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "af99a58898384ed98dd89a527f4af713"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "8bbb2bdd492213faa88dd5c1d6560fc7"
server_key_iv: "f4f78b41c596eac2c9a6841391e71026"
server_key_version: "encryptionkey@0"
I20260809 16:33:43.423622 28601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:43.442335 28601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:43.444698 28601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:43.444885 28601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:43.445169 28601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:43.445609 28601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:43.445667 28601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:43.445706 28601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:43.445740 28601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:43.453022 28601 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:39449
I20260809 16:33:43.453112 28823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:39449 every 8 connection(s)
I20260809 16:33:43.453612 28601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:33:43.460144 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 28601
I20260809 16:33:43.460250 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:33:43.460500 30814 external_mini_cluster.cc:1428] Setting key a19101f7630839d082a7ffebfc7c25ed
Aug 09 16:33:43 dist-test-slave-j2vl krb5kdc[28223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293223, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:33:43.465619 28354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.454466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:38307 (local address 127.30.23.190:33261)
0809 16:33:43.454542 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:43.454544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:43.454979 (+ 435us) server_negotiation.cc:407] Connection header received
0809 16:33:43.455579 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:43.455580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:43.455592 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:43.455651 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:43.456763 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.457483 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.458816 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.459014 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.459248 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.462005 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:43.462007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:43.462007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:43.462016 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:43.463933 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:43.464448 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:43.464450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:43.464450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:43.464522 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:43.464825 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:43.464826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:43.464826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:43.464966 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:43.465058 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:43.465412 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 16:33:43.465516 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:33:43.465929 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.454427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33261 (local address 127.30.23.130:38307)
0809 16:33:43.454606 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:43.454663 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:43.455457 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:43.455836 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:43.455840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:43.456449 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:43.456654 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.456659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.457593 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.457594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.458640 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.458644 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.459292 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.459294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.459429 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.459973 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:43.459982 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:43.461899 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:43.464055 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:43.464064 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:43.464067 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:43.464348 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:43.464608 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:43.464610 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:43.464610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:43.464745 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:43.465047 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:43.465055 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:43.465224 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 16:33:43.465354 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 16:33:43.465568 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260809 16:33:43.466559 28824 heartbeater.cc:347] Connected to a master server at 127.30.23.190:33261
I20260809 16:33:43.466696 28824 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:43.466965 28824 heartbeater.cc:511] Master 127.30.23.190:33261 requested a full tablet report, sending...
I20260809 16:33:43.467504 28291 ts_manager.cc:194] Registered new tserver with Master: af99a58898384ed98dd89a527f4af713 (127.30.23.130:39449)
I20260809 16:33:43.468096 28291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:38307
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:33:43.495218 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33261
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:44523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:43.586340 28832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:43.586568 28832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:43.586594 28832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:43.587771 28832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:43.587808 28832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:43.587837 28832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:33:43.589438 28832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:44523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:33261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:43.589665 28832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:43.589857 28832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:43.597183 28840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:43.597180 28842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:43.597267 28838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:43.597354 28832 server_base.cc:1034] running on GCE node
I20260809 16:33:43.597949 28832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:43.598260 28832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:43.599433 28832 hybrid_clock.cc:648] HybridClock initialized: now 1786293223599423 us; error 38 us; skew 500 ppm
Aug 09 16:33:43 dist-test-slave-j2vl krb5kdc[28223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293223, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:43.602373 28832 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:33:43.602839 28832 webserver.cc:460] Webserver started at http://127.30.23.131:34205/ using document root <none> and password file <none>
I20260809 16:33:43.603093 28832 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:43.603143 28832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:43.603281 28832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:43.604404 28832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "d14fb4cfe4d04e7593664ae6f39f0e06"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "5bbaf06c95bd0a68808be3337398cac3"
server_key_iv: "3468488f39fb6a9d1ed8c5ff861830f8"
server_key_version: "encryptionkey@0"
I20260809 16:33:43.604799 28832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "d14fb4cfe4d04e7593664ae6f39f0e06"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "5bbaf06c95bd0a68808be3337398cac3"
server_key_iv: "3468488f39fb6a9d1ed8c5ff861830f8"
server_key_version: "encryptionkey@0"
I20260809 16:33:43.606411 28832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:33:43.607216 28852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:43.607437 28832 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:43.607548 28832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "d14fb4cfe4d04e7593664ae6f39f0e06"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "5bbaf06c95bd0a68808be3337398cac3"
server_key_iv: "3468488f39fb6a9d1ed8c5ff861830f8"
server_key_version: "encryptionkey@0"
I20260809 16:33:43.607626 28832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:43.619387 28832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:43.622097 28832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:43.622308 28832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:43.622581 28832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:43.622959 28832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:43.623030 28832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:43.623081 28832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:43.623126 28832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:43.630453 28832 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:46879
I20260809 16:33:43.630492 29030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:46879 every 8 connection(s)
I20260809 16:33:43.630857 28832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:33:43.632344 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 28832
I20260809 16:33:43.632468 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:33:43.632640 30814 external_mini_cluster.cc:1428] Setting key 7190da46bf972042aaa1c91959b2e0e9
Aug 09 16:33:43 dist-test-slave-j2vl krb5kdc[28223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293223, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:33:43.642279 28354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.631749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:38825 (local address 127.30.23.190:33261)
0809 16:33:43.631806 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:43.631809 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:43.632036 (+ 227us) server_negotiation.cc:407] Connection header received
0809 16:33:43.632736 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:43.632738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:43.632752 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:43.632817 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:43.634118 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.634677 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.636246 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.636484 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.636567 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.639077 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:43.639079 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:43.639079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:43.639087 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:43.640637 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:43.641107 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:43.641109 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:43.641109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:43.641171 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:43.641608 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:43.641609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:43.641610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:43.641739 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:43.641806 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:43.642132 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 16:33:43.642209 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:33:43.642537 29038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.631712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:33261 (local address 127.30.23.131:38825)
0809 16:33:43.631912 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:43.631954 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:43.632614 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:43.632913 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:43.632918 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:43.633787 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:43.634006 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.634011 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.634802 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.634803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.636129 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.636132 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.636571 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.636572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.636711 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.637280 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:43.637290 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:43.638934 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:43.640768 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:43.640775 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:43.640777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:43.641015 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:43.641288 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:43.641289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:43.641289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:43.641476 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:43.641853 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:43.641860 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:43.641994 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 16:33:43.642088 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 16:33:43.642235 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260809 16:33:43.643213 29034 heartbeater.cc:347] Connected to a master server at 127.30.23.190:33261
I20260809 16:33:43.643349 29034 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:43.643584 29034 heartbeater.cc:511] Master 127.30.23.190:33261 requested a full tablet report, sending...
I20260809 16:33:43.644040 28288 ts_manager.cc:194] Registered new tserver with Master: d14fb4cfe4d04e7593664ae6f39f0e06 (127.30.23.131:46879)
I20260809 16:33:43.644510 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:33:43.644563 28288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:38825
I20260809 16:33:43.653457 28354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.646099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34568 (local address 127.30.23.190:33261)
0809 16:33:43.646171 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:43.646173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:43.646184 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:33:43.646264 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:43.646265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:43.646273 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:43.646328 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:43.646954 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.647443 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.648182 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.648409 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.648475 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.650258 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:43.650259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:43.650259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:43.650268 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:43.651610 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:43.652127 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:43.652128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:43.652128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:43.652205 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:43.652687 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:43.652688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:43.652689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:43.652892 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:43.652994 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:43.653217 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 16:33:43.653283 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 16:33:43.656134 28288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:33:43.658121 28288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:33:43.665088 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.660986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:46879 (local address 127.0.0.1:53178)
0809 16:33:43.661136 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:43.661180 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:43.661248 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:43.661602 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:43.661604 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:43.661606 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:43.661700 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.661703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.662969 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.662970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.663858 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.663860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.664767 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.664768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.664854 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.664870 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 16:33:43.664945 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:33:43.664999 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":103,"threads_started":1}
I20260809 16:33:43.665526 29056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.661087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53178 (local address 127.30.23.131:46879)
0809 16:33:43.661246 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:43.661253 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:43.661269 (+ 16us) server_negotiation.cc:407] Connection header received
0809 16:33:43.661395 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:43.661397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:43.661425 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:43.661508 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:43.661793 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.662859 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.663952 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.664689 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.664761 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.664831 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:43.665313 (+ 482us) server_negotiation.cc:299] Negotiation successful
0809 16:33:43.665426 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260809 16:33:43.665949 29054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.661238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:46739 (local address 127.0.0.1:34128)
0809 16:33:43.662245 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:43.662256 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:43.662309 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:43.662499 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:43.662500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:43.662502 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:43.662568 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.662570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.663779 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.663780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.664548 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.664550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.665518 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.665519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.665740 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.665746 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:33:43.665816 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 16:33:43.665866 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"thread_start_us":641,"threads_started":1}
I20260809 16:33:43.666131 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.661571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.30.23.130:39449)
0809 16:33:43.661982 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:43.661986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:43.661998 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:33:43.662036 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:43.662037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:43.662053 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:43.662143 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:43.662282 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.663377 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.664946 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.665750 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.665849 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.665904 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:43.666003 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 16:33:43.666058 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":79,"threads_started":1}
I20260809 16:33:43.666095 29057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.660987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:39449 (local address 127.0.0.1:60542)
0809 16:33:43.661409 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:43.661421 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:43.661478 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:43.662141 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:43.662142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:43.662145 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:43.662195 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.662196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.663513 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.663514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.664651 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.664654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.665860 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.665861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.665924 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.665928 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:43.665990 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:33:43.666029 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260809 16:33:43.666301 28932 tablet_service.cc:1467] Processing CreateTablet for tablet 856e3760386b4585a3cac523ae42fb6e (DEFAULT_TABLE table=test-table [id=f51100214bdd4e99a80aa78a18c9cef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:43.666728 28932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 856e3760386b4585a3cac523ae42fb6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:43.666887 28718 tablet_service.cc:1467] Processing CreateTablet for tablet 856e3760386b4585a3cac523ae42fb6e (DEFAULT_TABLE table=test-table [id=f51100214bdd4e99a80aa78a18c9cef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:43.667224 28718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 856e3760386b4585a3cac523ae42fb6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:43.669652 29058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.661544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34128 (local address 127.30.23.129:46739)
0809 16:33:43.661756 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:43.661761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:43.662312 (+ 551us) server_negotiation.cc:407] Connection header received
0809 16:33:43.662368 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:43.662370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:43.662387 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:43.662489 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:43.662638 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.663672 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.664644 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.665432 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.665499 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.665554 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:43.669520 (+ 3966us) server_negotiation.cc:299] Negotiation successful
0809 16:33:43.669575 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":104,"threads_started":1}
I20260809 16:33:43.670758 29061 tablet_bootstrap.cc:492] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713: Bootstrap starting.
I20260809 16:33:43.671060 29060 tablet_bootstrap.cc:492] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06: Bootstrap starting.
I20260809 16:33:43.671555 29061 tablet_bootstrap.cc:654] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:43.671466 28495 tablet_service.cc:1467] Processing CreateTablet for tablet 856e3760386b4585a3cac523ae42fb6e (DEFAULT_TABLE table=test-table [id=f51100214bdd4e99a80aa78a18c9cef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:43.671743 29060 tablet_bootstrap.cc:654] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:43.672122 28495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 856e3760386b4585a3cac523ae42fb6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:43.672206 29061 log.cc:824] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:43.672366 29060 log.cc:824] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:43.673112 29061 tablet_bootstrap.cc:492] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713: No bootstrap required, opened a new log
I20260809 16:33:43.673125 29060 tablet_bootstrap.cc:492] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06: No bootstrap required, opened a new log
I20260809 16:33:43.673195 29060 ts_tablet_manager.cc:1397] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:33:43.673195 29061 ts_tablet_manager.cc:1397] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:33:43.675278 29060 raft_consensus.cc:348] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af99a58898384ed98dd89a527f4af713" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 39449 } } peers { permanent_uuid: "d14fb4cfe4d04e7593664ae6f39f0e06" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 46879 } } peers { permanent_uuid: "7e360ae130a44121a891e5da0a369455" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46739 } }
I20260809 16:33:43.675273 29061 raft_consensus.cc:348] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af99a58898384ed98dd89a527f4af713" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 39449 } } peers { permanent_uuid: "d14fb4cfe4d04e7593664ae6f39f0e06" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 46879 } } peers { permanent_uuid: "7e360ae130a44121a891e5da0a369455" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46739 } }
I20260809 16:33:43.675479 29060 raft_consensus.cc:374] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:43.675479 29061 raft_consensus.cc:374] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:43.675529 29061 raft_consensus.cc:729] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af99a58898384ed98dd89a527f4af713, State: Initialized, Role: FOLLOWER
I20260809 16:33:43.675323 29066 tablet_bootstrap.cc:492] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455: Bootstrap starting.
I20260809 16:33:43.675529 29060 raft_consensus.cc:729] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14fb4cfe4d04e7593664ae6f39f0e06, State: Initialized, Role: FOLLOWER
I20260809 16:33:43.675662 29061 consensus_queue.cc:260] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af99a58898384ed98dd89a527f4af713" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 39449 } } peers { permanent_uuid: "d14fb4cfe4d04e7593664ae6f39f0e06" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 46879 } } peers { permanent_uuid: "7e360ae130a44121a891e5da0a369455" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46739 } }
I20260809 16:33:43.675693 29060 consensus_queue.cc:260] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af99a58898384ed98dd89a527f4af713" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 39449 } } peers { permanent_uuid: "d14fb4cfe4d04e7593664ae6f39f0e06" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 46879 } } peers { permanent_uuid: "7e360ae130a44121a891e5da0a369455" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46739 } }
I20260809 16:33:43.675988 29060 ts_tablet_manager.cc:1428] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:33:43.675985 29061 ts_tablet_manager.cc:1428] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:33:43.676119 29034 heartbeater.cc:503] Master 127.30.23.190:33261 was elected leader, sending a full tablet report...
I20260809 16:33:43.676186 28824 heartbeater.cc:503] Master 127.30.23.190:33261 was elected leader, sending a full tablet report...
I20260809 16:33:43.676263 29066 tablet_bootstrap.cc:654] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:43.676894 29066 log.cc:824] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:43.677805 29066 tablet_bootstrap.cc:492] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455: No bootstrap required, opened a new log
I20260809 16:33:43.677865 29066 ts_tablet_manager.cc:1397] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:33:43.679414 29066 raft_consensus.cc:348] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af99a58898384ed98dd89a527f4af713" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 39449 } } peers { permanent_uuid: "d14fb4cfe4d04e7593664ae6f39f0e06" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 46879 } } peers { permanent_uuid: "7e360ae130a44121a891e5da0a369455" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46739 } }
I20260809 16:33:43.679566 29066 raft_consensus.cc:374] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:43.679591 29066 raft_consensus.cc:729] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e360ae130a44121a891e5da0a369455, State: Initialized, Role: FOLLOWER
I20260809 16:33:43.679714 29066 consensus_queue.cc:260] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af99a58898384ed98dd89a527f4af713" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 39449 } } peers { permanent_uuid: "d14fb4cfe4d04e7593664ae6f39f0e06" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 46879 } } peers { permanent_uuid: "7e360ae130a44121a891e5da0a369455" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46739 } }
I20260809 16:33:43.680039 29066 ts_tablet_manager.cc:1428] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:33:43.680076 28596 heartbeater.cc:503] Master 127.30.23.190:33261 was elected leader, sending a full tablet report...
W20260809 16:33:43.704425 28825 tablet.cc:2369] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:33:43.761291 28597 tablet.cc:2369] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:33:43.765419 29073 raft_consensus.cc:482] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:33:43.765600 29073 raft_consensus.cc:504] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af99a58898384ed98dd89a527f4af713" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 39449 } } peers { permanent_uuid: "d14fb4cfe4d04e7593664ae6f39f0e06" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 46879 } } peers { permanent_uuid: "7e360ae130a44121a891e5da0a369455" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46739 } }
I20260809 16:33:43.766023 29073 leader_election.cc:288] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af99a58898384ed98dd89a527f4af713 (127.30.23.130:39449), d14fb4cfe4d04e7593664ae6f39f0e06 (127.30.23.131:46879)
I20260809 16:33:43.770460 29056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.766398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:45587 (local address 127.30.23.131:46879)
0809 16:33:43.766452 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:43.766454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:43.766802 (+ 348us) server_negotiation.cc:407] Connection header received
0809 16:33:43.766874 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:43.766875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:43.766887 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:43.766972 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:43.767125 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.767785 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.769200 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.769856 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.769948 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.769957 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:43.770317 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 16:33:43.770380 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:33:43.770460 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.766424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:53163 (local address 127.30.23.130:39449)
0809 16:33:43.766481 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:43.766483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:43.766491 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:33:43.766528 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:43.766529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:43.766544 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:43.766619 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:43.766828 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.767632 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.769200 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.769852 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.769946 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.769957 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:43.770340 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 16:33:43.770394 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:33:43.770467 28598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.766276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:39449 (local address 127.30.23.129:53163)
0809 16:33:43.766343 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:43.766354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:43.766416 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:43.766625 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:43.766626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:43.766628 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:43.766711 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.766713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.767748 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.767748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.769081 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.769084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.769976 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.769978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.770076 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.770078 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:33:43.770293 (+ 215us) client_negotiation.cc:240] Negotiation successful
0809 16:33:43.770335 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":58}
I20260809 16:33:43.770478 29075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.766276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:46879 (local address 127.30.23.129:45587)
0809 16:33:43.766671 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:43.766683 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:43.766740 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:43.766980 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:43.766981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:43.766982 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:43.767047 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.767048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.767874 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.767875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.769082 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:43.769084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.769977 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.769978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:43.770071 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.770075 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:43.770274 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 16:33:43.770332 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":5,"thread_start_us":68,"threads_started":1}
I20260809 16:33:43.770910 28747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "856e3760386b4585a3cac523ae42fb6e" candidate_uuid: "7e360ae130a44121a891e5da0a369455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af99a58898384ed98dd89a527f4af713" is_pre_election: true
I20260809 16:33:43.771114 28747 raft_consensus.cc:2454] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e360ae130a44121a891e5da0a369455 in term 0.
I20260809 16:33:43.771126 28964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "856e3760386b4585a3cac523ae42fb6e" candidate_uuid: "7e360ae130a44121a891e5da0a369455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14fb4cfe4d04e7593664ae6f39f0e06" is_pre_election: true
I20260809 16:33:43.771271 28964 raft_consensus.cc:2454] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e360ae130a44121a891e5da0a369455 in term 0.
I20260809 16:33:43.771540 28412 leader_election.cc:302] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e360ae130a44121a891e5da0a369455, af99a58898384ed98dd89a527f4af713; no voters:
I20260809 16:33:43.771698 29073 raft_consensus.cc:2781] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:33:43.771760 29073 raft_consensus.cc:482] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:33:43.771813 29073 raft_consensus.cc:3037] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:43.772622 29073 raft_consensus.cc:504] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af99a58898384ed98dd89a527f4af713" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 39449 } } peers { permanent_uuid: "d14fb4cfe4d04e7593664ae6f39f0e06" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 46879 } } peers { permanent_uuid: "7e360ae130a44121a891e5da0a369455" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46739 } }
I20260809 16:33:43.772792 29073 leader_election.cc:288] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [CANDIDATE]: Term 1 election: Requested vote from peers af99a58898384ed98dd89a527f4af713 (127.30.23.130:39449), d14fb4cfe4d04e7593664ae6f39f0e06 (127.30.23.131:46879)
I20260809 16:33:43.772966 28747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "856e3760386b4585a3cac523ae42fb6e" candidate_uuid: "7e360ae130a44121a891e5da0a369455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af99a58898384ed98dd89a527f4af713"
I20260809 16:33:43.772970 28964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "856e3760386b4585a3cac523ae42fb6e" candidate_uuid: "7e360ae130a44121a891e5da0a369455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14fb4cfe4d04e7593664ae6f39f0e06"
I20260809 16:33:43.773067 28964 raft_consensus.cc:3037] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:43.773072 28747 raft_consensus.cc:3037] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:43.773910 28747 raft_consensus.cc:2454] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e360ae130a44121a891e5da0a369455 in term 1.
I20260809 16:33:43.773953 28964 raft_consensus.cc:2454] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e360ae130a44121a891e5da0a369455 in term 1.
I20260809 16:33:43.774148 28415 leader_election.cc:302] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e360ae130a44121a891e5da0a369455, d14fb4cfe4d04e7593664ae6f39f0e06; no voters:
I20260809 16:33:43.774278 29073 raft_consensus.cc:2781] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:43.774484 29073 raft_consensus.cc:686] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [term 1 LEADER]: Becoming Leader. State: Replica: 7e360ae130a44121a891e5da0a369455, State: Running, Role: LEADER
I20260809 16:33:43.774603 29073 consensus_queue.cc:237] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af99a58898384ed98dd89a527f4af713" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 39449 } } peers { permanent_uuid: "d14fb4cfe4d04e7593664ae6f39f0e06" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 46879 } } peers { permanent_uuid: "7e360ae130a44121a891e5da0a369455" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46739 } }
I20260809 16:33:43.775764 28291 catalog_manager.cc:5495] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e360ae130a44121a891e5da0a369455 (127.30.23.129). New cstate: current_term: 1 leader_uuid: "7e360ae130a44121a891e5da0a369455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af99a58898384ed98dd89a527f4af713" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 39449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d14fb4cfe4d04e7593664ae6f39f0e06" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 46879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e360ae130a44121a891e5da0a369455" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 46739 } health_report { overall_health: HEALTHY } } }
I20260809 16:33:43.785181 29058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.782726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34130 (local address 127.30.23.129:46739)
0809 16:33:43.782842 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:43.782844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:43.782853 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:33:43.782895 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:43.782896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:43.782914 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:43.782999 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:33:43.783155 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.783639 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.784399 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:43.784595 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:43.784675 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:43.784842 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:33:43.784938 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:33:43.785054 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:43.785086 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 16:33:43.785098 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 16:33:43.787142 28964 raft_consensus.cc:1261] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06 [term 1 FOLLOWER]: Refusing update from remote peer 7e360ae130a44121a891e5da0a369455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:43.787158 28747 raft_consensus.cc:1261] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713 [term 1 FOLLOWER]: Refusing update from remote peer 7e360ae130a44121a891e5da0a369455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:43.787581 29076 consensus_queue.cc:1035] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d14fb4cfe4d04e7593664ae6f39f0e06" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 46879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:43.787717 29073 consensus_queue.cc:1035] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af99a58898384ed98dd89a527f4af713" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:43.790452 29079 mvcc.cc:205] Tried to move back new op lower bound from 7316657044629180416 to 7316657044581564416. Current Snapshot: MvccSnapshot[applied={T|T < 7316657044629180416}]
I20260809 16:33:43.790537 29081 mvcc.cc:205] Tried to move back new op lower bound from 7316657044629180416 to 7316657044581564416. Current Snapshot: MvccSnapshot[applied={T|T < 7316657044629180416}]
I20260809 16:33:43.793624 28291 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:43.793723 28291 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:43.794790 28291 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3dd41f572e54038a5361a3ab70216ba: Sending DeleteTablet for 3 replicas of tablet 856e3760386b4585a3cac523ae42fb6e
I20260809 16:33:43.795274 28932 tablet_service.cc:1514] Processing DeleteTablet for tablet 856e3760386b4585a3cac523ae42fb6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:43 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:53178
I20260809 16:33:43.795310 28718 tablet_service.cc:1514] Processing DeleteTablet for tablet 856e3760386b4585a3cac523ae42fb6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:43 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:60542
I20260809 16:33:43.795320 28495 tablet_service.cc:1514] Processing DeleteTablet for tablet 856e3760386b4585a3cac523ae42fb6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:43 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:34128
I20260809 16:33:43.795555 29098 tablet_replica.cc:330] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455: stopping tablet replica
I20260809 16:33:43.795667 29098 raft_consensus.cc:2229] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:33:43.795684 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 28390
I20260809 16:33:43.795800 29098 raft_consensus.cc:2258] T 856e3760386b4585a3cac523ae42fb6e P 7e360ae130a44121a891e5da0a369455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:43.796281 29097 tablet_replica.cc:330] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06: stopping tablet replica
I20260809 16:33:43.796370 29097 raft_consensus.cc:2229] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:43.796450 29097 raft_consensus.cc:2258] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:43.798856 29092 tablet_replica.cc:330] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713: stopping tablet replica
I20260809 16:33:43.798957 29092 raft_consensus.cc:2229] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:43.799036 29092 raft_consensus.cc:2258] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:43.802064 29092 ts_tablet_manager.cc:1905] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:43.804010 29092 ts_tablet_manager.cc:1918] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:43.804086 29092 log.cc:1196] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/856e3760386b4585a3cac523ae42fb6e
I20260809 16:33:43.804334 29092 ts_tablet_manager.cc:1939] T 856e3760386b4585a3cac523ae42fb6e P af99a58898384ed98dd89a527f4af713: Deleting consensus metadata
I20260809 16:33:43.806846 28268 catalog_manager.cc:4841] TS af99a58898384ed98dd89a527f4af713 (127.30.23.130:39449): tablet 856e3760386b4585a3cac523ae42fb6e (table test-table [id=f51100214bdd4e99a80aa78a18c9cef1]) successfully deleted
I20260809 16:33:43.807251 29097 ts_tablet_manager.cc:1905] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:43.808990 29097 ts_tablet_manager.cc:1918] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:43.809067 29097 log.cc:1196] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/856e3760386b4585a3cac523ae42fb6e
I20260809 16:33:43.809461 29097 ts_tablet_manager.cc:1939] T 856e3760386b4585a3cac523ae42fb6e P d14fb4cfe4d04e7593664ae6f39f0e06: Deleting consensus metadata
I20260809 16:33:43.810849 28269 catalog_manager.cc:4841] TS d14fb4cfe4d04e7593664ae6f39f0e06 (127.30.23.131:46879): tablet 856e3760386b4585a3cac523ae42fb6e (table test-table [id=f51100214bdd4e99a80aa78a18c9cef1]) successfully deleted
W20260809 16:33:43.813277 28268 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:46739 (error 108)
I20260809 16:33:43.813728 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 28601
I20260809 16:33:43.825680 29054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:43.813867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:46739 (local address 127.0.0.1:34146)
0809 16:33:43.825523 (+ 11656us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:43.825617 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:46739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11616}
W20260809 16:33:43.829931 28268 catalog_manager.cc:4584] TS 7e360ae130a44121a891e5da0a369455 (127.30.23.129:46739): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 856e3760386b4585a3cac523ae42fb6e: Network error: Client connection negotiation failed: client connection to 127.30.23.129:46739: connect: Connection refused (error 111)
I20260809 16:33:43.834792 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 28832
I20260809 16:33:43.854662 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 28246
2026-08-09T16:33:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2370 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T16:33:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:33:43Z Disabled control of system clock
I20260809 16:33:43.899085 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:34147
--webserver_interface=127.30.23.190
--webserver_port=0
--builtin_ntp_servers=127.30.23.148:34679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:34147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:33:43.988585 29105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:43.988817 29105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:43.988840 29105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:43.989960 29105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:43.989998 29105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:43.990011 29105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:43.990024 29105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:43.990037 29105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:43.991333 29105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:34679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:34147
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:34147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:43.991545 29105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:43.991798 29105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:43.997329 29113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:43.997255 29111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:43.997255 29118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:43.997558 29105 server_base.cc:1034] running on GCE node
I20260809 16:33:43.997931 29105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:43.998220 29105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:43.999437 29105 hybrid_clock.cc:648] HybridClock initialized: now 1786293223999400 us; error 55 us; skew 500 ppm
I20260809 16:33:44.000806 29105 webserver.cc:460] Webserver started at http://127.30.23.190:35097/ using document root <none> and password file <none>
I20260809 16:33:44.001062 29105 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:44.001111 29105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:44.001217 29105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:44.002502 29105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "a5eaee9158164adf83bd89fa9ac9c47d"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "258cc0f75a1a1941b0131d7606191ea5"
server_key_iv: "2b3ed7abc0f49559e559ed85d63334df"
server_key_version: "encryptionkey@0"
I20260809 16:33:44.002923 29105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "a5eaee9158164adf83bd89fa9ac9c47d"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "258cc0f75a1a1941b0131d7606191ea5"
server_key_iv: "2b3ed7abc0f49559e559ed85d63334df"
server_key_version: "encryptionkey@0"
I20260809 16:33:44.005147 29105 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 16:33:44.006282 29125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:44.006755 29105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:44.006866 29105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "a5eaee9158164adf83bd89fa9ac9c47d"
format_stamp: "Formatted at 2026-08-09 16:33:43 on dist-test-slave-j2vl"
server_key: "258cc0f75a1a1941b0131d7606191ea5"
server_key_iv: "2b3ed7abc0f49559e559ed85d63334df"
server_key_version: "encryptionkey@0"
I20260809 16:33:44.006929 29105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:44.019397 29105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:44.019712 29105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:44.019820 29105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:44.024281 29105 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:34147
I20260809 16:33:44.024374 29201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:34147 every 8 connection(s)
I20260809 16:33:44.024716 29105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:44.025888 29202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:44.028322 29202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d: Bootstrap starting.
I20260809 16:33:44.028949 29202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:44.029276 29202 log.cc:824] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:44.030144 29202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d: No bootstrap required, opened a new log
I20260809 16:33:44.031832 29202 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5eaee9158164adf83bd89fa9ac9c47d" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34147 } }
I20260809 16:33:44.032002 29202 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:44.032049 29202 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5eaee9158164adf83bd89fa9ac9c47d, State: Initialized, Role: FOLLOWER
I20260809 16:33:44.032246 29202 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5eaee9158164adf83bd89fa9ac9c47d" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34147 } }
I20260809 16:33:44.032359 29202 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:44.032402 29202 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:44.032455 29202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:44.033010 29202 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5eaee9158164adf83bd89fa9ac9c47d" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34147 } }
I20260809 16:33:44.033135 29202 leader_election.cc:302] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5eaee9158164adf83bd89fa9ac9c47d; no voters:
I20260809 16:33:44.033382 29202 leader_election.cc:288] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:33:44.033509 29207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:44.033651 29202 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:44.033701 29207 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [term 1 LEADER]: Becoming Leader. State: Replica: a5eaee9158164adf83bd89fa9ac9c47d, State: Running, Role: LEADER
I20260809 16:33:44.033840 29207 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5eaee9158164adf83bd89fa9ac9c47d" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34147 } }
I20260809 16:33:44.034300 29208 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5eaee9158164adf83bd89fa9ac9c47d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5eaee9158164adf83bd89fa9ac9c47d" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34147 } } }
I20260809 16:33:44.034386 29208 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:44.034394 29209 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5eaee9158164adf83bd89fa9ac9c47d. Latest consensus state: current_term: 1 leader_uuid: "a5eaee9158164adf83bd89fa9ac9c47d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5eaee9158164adf83bd89fa9ac9c47d" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34147 } } }
I20260809 16:33:44.034631 29209 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:44.034889 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 29105
I20260809 16:33:44.035013 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:33:44.035252 30814 external_mini_cluster.cc:1428] Setting key 0fa6eadd7030336b9a39375c2c33348f
W20260809 16:33:44.035665 29228 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:33:44.035728 29228 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:33:44.035827 29219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:44.035995 29219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:44.037786 29219 catalog_manager.cc:1347] Generated new cluster ID: 0d42f1cd469a44d5932c55e708022583
I20260809 16:33:44.037840 29219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:33:44.041555 29232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.035798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39926 (local address 127.30.23.190:34147)
0809 16:33:44.035997 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:44.036007 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:44.036224 (+ 217us) server_negotiation.cc:407] Connection header received
0809 16:33:44.036348 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:44.036356 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:44.036427 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:44.036575 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:44.036801 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.038142 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.039260 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.039883 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.039999 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.040783 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:44.040787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:44.040788 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:44.040994 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:44.041035 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:44.041049 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:44.041115 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:44.041215 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 16:33:44.041297 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260809 16:33:44.054168 29219 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:33:44.055109 29219 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:44.068149 29219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d: Generated new TSK 0
I20260809 16:33:44.068392 29219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:33:44.070827 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34147
--builtin_ntp_servers=127.30.23.148:34679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:33:44.177789 29234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:44.178023 29234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:44.178047 29234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:44.179277 29234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:44.179332 29234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:44.179359 29234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:33:44.180882 29234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:34679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:44.181180 29234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:44.181578 29234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:44.187080 29241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:44.187090 29242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:44.187090 29245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:44.187987 29234 server_base.cc:1034] running on GCE node
I20260809 16:33:44.188251 29234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:44.188586 29234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:44.189774 29234 hybrid_clock.cc:648] HybridClock initialized: now 1786293224189745 us; error 44 us; skew 500 ppm
I20260809 16:33:44.191244 29234 webserver.cc:460] Webserver started at http://127.30.23.129:37579/ using document root <none> and password file <none>
I20260809 16:33:44.191547 29234 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:44.191599 29234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:44.191740 29234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:44.192862 29234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "d0ffccf80626424eba7f25baeeb03035"
format_stamp: "Formatted at 2026-08-09 16:33:44 on dist-test-slave-j2vl"
server_key: "fea927e97b04da73ccf124aa2c86a47c"
server_key_iv: "b857a3213b6576cba2ba65cc982f742c"
server_key_version: "encryptionkey@0"
I20260809 16:33:44.193274 29234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "d0ffccf80626424eba7f25baeeb03035"
format_stamp: "Formatted at 2026-08-09 16:33:44 on dist-test-slave-j2vl"
server_key: "fea927e97b04da73ccf124aa2c86a47c"
server_key_iv: "b857a3213b6576cba2ba65cc982f742c"
server_key_version: "encryptionkey@0"
I20260809 16:33:44.194800 29234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:44.195801 29253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:44.195986 29234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:44.196103 29234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "d0ffccf80626424eba7f25baeeb03035"
format_stamp: "Formatted at 2026-08-09 16:33:44 on dist-test-slave-j2vl"
server_key: "fea927e97b04da73ccf124aa2c86a47c"
server_key_iv: "b857a3213b6576cba2ba65cc982f742c"
server_key_version: "encryptionkey@0"
I20260809 16:33:44.196185 29234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:44.214154 29234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:44.214553 29234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:44.214737 29234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:44.214998 29234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:44.215371 29234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:44.215413 29234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:44.215437 29234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:44.215484 29234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:44.221980 29234 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:39803
I20260809 16:33:44.222096 29447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:39803 every 8 connection(s)
I20260809 16:33:44.222440 29234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:33:44.228336 29232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.223325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:59317 (local address 127.30.23.190:34147)
0809 16:33:44.223419 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:44.223421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:44.223624 (+ 203us) server_negotiation.cc:407] Connection header received
0809 16:33:44.224150 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:44.224151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:44.224166 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:44.224237 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:44.225044 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.225643 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.226747 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.226956 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.227027 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.228031 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:44.228033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:44.228033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:44.228038 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:44.228053 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:44.228057 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:44.228105 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:44.228203 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 16:33:44.228234 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 16:33:44.228463 29451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.223290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34147 (local address 127.30.23.129:59317)
0809 16:33:44.223490 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:44.223532 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:44.224032 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:44.224364 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:44.224368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:44.224376 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:44.224938 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.224943 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.225769 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.225770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.226613 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.226616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.227070 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.227071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.227202 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.227850 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:33:44.227851 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:44.227881 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:33:44.227907 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:44.228112 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:44.228115 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:44.228118 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:33:44.228166 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 16:33:44.228254 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260809 16:33:44.229203 29449 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34147
I20260809 16:33:44.229327 29449 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:44.229576 29449 heartbeater.cc:511] Master 127.30.23.190:34147 requested a full tablet report, sending...
I20260809 16:33:44.230189 29153 ts_manager.cc:194] Registered new tserver with Master: d0ffccf80626424eba7f25baeeb03035 (127.30.23.129:39803)
I20260809 16:33:44.230566 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 29234
I20260809 16:33:44.230679 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:33:44.230897 30814 external_mini_cluster.cc:1428] Setting key d4830dc3512ef059e6db0e8006ac8e56
I20260809 16:33:44.230917 29153 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.23.129:59317
I20260809 16:33:44.231869 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34147
--builtin_ntp_servers=127.30.23.148:34679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:33:44.320240 29453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:44.320474 29453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:44.320498 29453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:44.321758 29453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:44.321812 29453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:44.321841 29453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:33:44.323475 29453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:34679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:44.323786 29453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:44.324146 29453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:44.329880 29461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:44.329957 29459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:44.329905 29466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:44.330221 29453 server_base.cc:1034] running on GCE node
I20260809 16:33:44.330612 29453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:44.330929 29453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:44.332103 29453 hybrid_clock.cc:648] HybridClock initialized: now 1786293224332090 us; error 32 us; skew 500 ppm
I20260809 16:33:44.333774 29453 webserver.cc:460] Webserver started at http://127.30.23.130:41631/ using document root <none> and password file <none>
I20260809 16:33:44.334132 29453 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:44.334208 29453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:44.334364 29453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:44.335789 29453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "d27549814fc94315b4aff0f038c991ef"
format_stamp: "Formatted at 2026-08-09 16:33:44 on dist-test-slave-j2vl"
server_key: "56d3f01c4130c4e95124e0c79b2faa95"
server_key_iv: "516d9550c4af3fc3110d8790abc63d15"
server_key_version: "encryptionkey@0"
I20260809 16:33:44.336187 29453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "d27549814fc94315b4aff0f038c991ef"
format_stamp: "Formatted at 2026-08-09 16:33:44 on dist-test-slave-j2vl"
server_key: "56d3f01c4130c4e95124e0c79b2faa95"
server_key_iv: "516d9550c4af3fc3110d8790abc63d15"
server_key_version: "encryptionkey@0"
I20260809 16:33:44.337996 29453 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260809 16:33:44.339083 29473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:44.339313 29453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:44.339432 29453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "d27549814fc94315b4aff0f038c991ef"
format_stamp: "Formatted at 2026-08-09 16:33:44 on dist-test-slave-j2vl"
server_key: "56d3f01c4130c4e95124e0c79b2faa95"
server_key_iv: "516d9550c4af3fc3110d8790abc63d15"
server_key_version: "encryptionkey@0"
I20260809 16:33:44.339540 29453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:44.363859 29453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:44.364167 29453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:44.364279 29453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:44.364480 29453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:44.364857 29453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:44.364894 29453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:44.364940 29453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:44.364966 29453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:44.371742 29453 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:40623
I20260809 16:33:44.371778 29663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:40623 every 8 connection(s)
I20260809 16:33:44.372160 29453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:33:44.379048 29232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.373018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:53333 (local address 127.30.23.190:34147)
0809 16:33:44.373099 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:44.373102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:44.373335 (+ 233us) server_negotiation.cc:407] Connection header received
0809 16:33:44.374065 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:44.374067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:44.374078 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:44.374128 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:44.375095 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.375623 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.377104 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.377332 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.377576 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.378714 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:44.378715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:44.378715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:44.378721 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:44.378736 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:44.378740 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:44.378798 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:44.378907 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 16:33:44.378943 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 16:33:44.379086 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 29453
I20260809 16:33:44.379201 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:33:44.379208 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.372969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34147 (local address 127.30.23.130:53333)
0809 16:33:44.373207 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:44.373245 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:44.373880 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:44.374188 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:44.374191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:44.374200 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:44.374755 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.374760 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.375757 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.375758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.376992 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.376995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.377598 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.377599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.377749 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.378515 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:33:44.378522 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:44.378554 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:33:44.378580 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:44.378808 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:44.378813 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:44.378816 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:33:44.378882 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 16:33:44.378976 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":119,"threads_started":1}
I20260809 16:33:44.379413 30814 external_mini_cluster.cc:1428] Setting key 7cf9da366b1aeec37b0ecaedb10580bf
I20260809 16:33:44.379840 29665 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34147
I20260809 16:33:44.379951 29665 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:44.380184 29665 heartbeater.cc:511] Master 127.30.23.190:34147 requested a full tablet report, sending...
I20260809 16:33:44.380537 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34147
--builtin_ntp_servers=127.30.23.148:34679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:33:44.380780 29153 ts_manager.cc:194] Registered new tserver with Master: d27549814fc94315b4aff0f038c991ef (127.30.23.130:40623)
I20260809 16:33:44.381157 29153 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.23.130:53333
W20260809 16:33:44.479076 29669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:44.479317 29669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:44.479342 29669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:44.480512 29669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:44.480549 29669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:44.480576 29669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:33:44.482136 29669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:34679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:44.482389 29669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:44.482671 29669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:44.488692 29683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:44.488648 29669 server_base.cc:1034] running on GCE node
W20260809 16:33:44.488638 29685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:44.488621 29688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:44.489197 29669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:44.489539 29669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:44.490723 29669 hybrid_clock.cc:648] HybridClock initialized: now 1786293224490709 us; error 46 us; skew 500 ppm
I20260809 16:33:44.492173 29669 webserver.cc:460] Webserver started at http://127.30.23.131:39707/ using document root <none> and password file <none>
I20260809 16:33:44.492462 29669 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:44.492539 29669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:44.492702 29669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:44.493911 29669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "ada4948894fb4a12a8203ba223ba46aa"
format_stamp: "Formatted at 2026-08-09 16:33:44 on dist-test-slave-j2vl"
server_key: "76279f01d989368bac9b6135ce300f09"
server_key_iv: "70e18c2ca61458395aa93a428daa102f"
server_key_version: "encryptionkey@0"
I20260809 16:33:44.494369 29669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "ada4948894fb4a12a8203ba223ba46aa"
format_stamp: "Formatted at 2026-08-09 16:33:44 on dist-test-slave-j2vl"
server_key: "76279f01d989368bac9b6135ce300f09"
server_key_iv: "70e18c2ca61458395aa93a428daa102f"
server_key_version: "encryptionkey@0"
I20260809 16:33:44.496179 29669 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 16:33:44.497188 29705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:44.497444 29669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:44.497582 29669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "ada4948894fb4a12a8203ba223ba46aa"
format_stamp: "Formatted at 2026-08-09 16:33:44 on dist-test-slave-j2vl"
server_key: "76279f01d989368bac9b6135ce300f09"
server_key_iv: "70e18c2ca61458395aa93a428daa102f"
server_key_version: "encryptionkey@0"
I20260809 16:33:44.497670 29669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:44.512341 29669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:44.512715 29669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:44.512887 29669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:44.513154 29669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:44.513589 29669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:44.513648 29669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:44.513693 29669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:44.513725 29669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:44.520592 29669 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:34145
I20260809 16:33:44.520632 29891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:34145 every 8 connection(s)
I20260809 16:33:44.521155 29669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:33:44.527643 29232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.522208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:55921 (local address 127.30.23.190:34147)
0809 16:33:44.522272 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:44.522275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:44.522736 (+ 461us) server_negotiation.cc:407] Connection header received
0809 16:33:44.523185 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:44.523186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:44.523198 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:44.523266 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:44.524147 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.524667 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.525862 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.526142 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.526212 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.527300 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:44.527302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:44.527302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:44.527309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:44.527334 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:44.527340 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:44.527414 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:44.527543 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 16:33:44.527578 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:33:44.527880 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.522148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34147 (local address 127.30.23.131:55921)
0809 16:33:44.522479 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:44.522528 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:44.523070 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:44.523386 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:44.523390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:44.523399 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:44.524003 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.524008 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.524786 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.524787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.525683 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.525686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.526258 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.526259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.526452 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.527082 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:33:44.527082 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:44.527115 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:33:44.527142 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:44.527435 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:44.527444 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:44.527448 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:44.527538 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 16:33:44.527634 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":212,"threads_started":1}
I20260809 16:33:44.528136 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 29669
I20260809 16:33:44.528266 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:33:44.528528 30814 external_mini_cluster.cc:1428] Setting key 5c0db52bf3a31ca186b14b1fe41a2523
I20260809 16:33:44.528539 29893 heartbeater.cc:347] Connected to a master server at 127.30.23.190:34147
I20260809 16:33:44.528662 29893 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:44.528910 29893 heartbeater.cc:511] Master 127.30.23.190:34147 requested a full tablet report, sending...
I20260809 16:33:44.529336 29152 ts_manager.cc:194] Registered new tserver with Master: ada4948894fb4a12a8203ba223ba46aa (127.30.23.131:34145)
I20260809 16:33:44.529827 29152 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.23.131:55921
I20260809 16:33:44.541105 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:33:44.546247 29232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.542702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39938 (local address 127.30.23.190:34147)
0809 16:33:44.542788 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:44.542791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:44.542914 (+ 123us) server_negotiation.cc:407] Connection header received
0809 16:33:44.542946 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:44.542946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:44.542961 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:44.543020 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:44.543240 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.543758 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.544750 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.544943 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.545008 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.545819 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:44.545821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:44.545821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:44.545827 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:44.545841 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:44.545845 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:44.545928 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:44.546126 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 16:33:44.546142 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 16:33:44.548192 29152 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:33:44.549988 29152 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:33:44.557408 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.553101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45176 (local address 127.30.23.130:40623)
0809 16:33:44.553535 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:44.553542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:44.553554 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:33:44.553601 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:44.553603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:44.553619 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:44.553695 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:44.553863 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.554662 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.556321 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.557041 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.557107 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.557207 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:44.557242 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 16:33:44.557290 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":93,"threads_started":1}
I20260809 16:33:44.557411 29910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.552734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41320 (local address 127.30.23.131:34145)
0809 16:33:44.553230 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:44.553234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:44.553245 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:33:44.553288 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:44.553290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:44.553306 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:44.553615 (+ 309us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:44.553836 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.554923 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.556147 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.556922 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.557007 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.557117 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:44.557243 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 16:33:44.557283 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":98,"threads_started":1}
I20260809 16:33:44.557654 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.552863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:40623 (local address 127.0.0.1:45176)
0809 16:33:44.553035 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:44.553047 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:44.553101 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:44.553764 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:44.553765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:44.553767 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:44.553802 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.553804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.554779 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.554780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.556222 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.556225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.557127 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.557128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.557182 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.557186 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:44.557228 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 16:33:44.557280 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":113,"thread_start_us":62,"threads_started":1}
I20260809 16:33:44.557654 29911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.552615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:34145 (local address 127.0.0.1:41320)
0809 16:33:44.552961 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:44.552975 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:44.553030 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:44.553699 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:44.553701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:44.553703 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:44.553752 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.553753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.555027 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.555028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.556047 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.556050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.557020 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.557021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.557111 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.557129 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 16:33:44.557210 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:33:44.557280 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":8,"thread_start_us":60,"threads_started":1}
I20260809 16:33:44.558707 29909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.552609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:39803 (local address 127.0.0.1:34562)
0809 16:33:44.552786 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:44.552826 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:44.552909 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:44.553346 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:44.553349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:44.553351 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:44.553459 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.553461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.554983 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.554984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.556222 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.556225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.558447 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.558447 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.558524 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.558528 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:44.558593 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 16:33:44.558639 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":15,"thread_start_us":80,"threads_started":1}
I20260809 16:33:44.558741 29912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.552807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.30.23.129:39803)
0809 16:33:44.553124 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:44.553129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:44.553139 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:33:44.553207 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:44.553209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:44.553226 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:44.553624 (+ 398us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:44.553696 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.554889 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.557546 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.558372 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.558448 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.558535 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:44.558610 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 16:33:44.558648 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":118,"threads_started":1}
I20260809 16:33:44.558624 29550 tablet_service.cc:1467] Processing CreateTablet for tablet adcf75f230a74b2a82c1f3d98b160df7 (DEFAULT_TABLE table=test-table [id=aefc8dd385da4d5eae22e86dd8ec3997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:44.558745 29784 tablet_service.cc:1467] Processing CreateTablet for tablet adcf75f230a74b2a82c1f3d98b160df7 (DEFAULT_TABLE table=test-table [id=aefc8dd385da4d5eae22e86dd8ec3997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:44.559093 29784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adcf75f230a74b2a82c1f3d98b160df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:44.559171 29550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adcf75f230a74b2a82c1f3d98b160df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:44.561949 29914 tablet_bootstrap.cc:492] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa: Bootstrap starting.
I20260809 16:33:44.562605 29914 tablet_bootstrap.cc:654] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:44.562507 29338 tablet_service.cc:1467] Processing CreateTablet for tablet adcf75f230a74b2a82c1f3d98b160df7 (DEFAULT_TABLE table=test-table [id=aefc8dd385da4d5eae22e86dd8ec3997]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:44.562786 29338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adcf75f230a74b2a82c1f3d98b160df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:44.562834 29914 log.cc:824] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:44.563162 29916 tablet_bootstrap.cc:492] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef: Bootstrap starting.
I20260809 16:33:44.563639 29914 tablet_bootstrap.cc:492] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa: No bootstrap required, opened a new log
I20260809 16:33:44.563700 29916 tablet_bootstrap.cc:654] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:44.563701 29914 ts_tablet_manager.cc:1397] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:33:44.563925 29916 log.cc:824] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:44.564558 29916 tablet_bootstrap.cc:492] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef: No bootstrap required, opened a new log
I20260809 16:33:44.564648 29916 ts_tablet_manager.cc:1397] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:33:44.566284 29921 tablet_bootstrap.cc:492] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035: Bootstrap starting.
I20260809 16:33:44.566489 29914 raft_consensus.cc:348] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada4948894fb4a12a8203ba223ba46aa" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34145 } } peers { permanent_uuid: "d0ffccf80626424eba7f25baeeb03035" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 39803 } } peers { permanent_uuid: "d27549814fc94315b4aff0f038c991ef" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40623 } }
I20260809 16:33:44.566735 29914 raft_consensus.cc:374] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:44.566785 29914 raft_consensus.cc:729] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada4948894fb4a12a8203ba223ba46aa, State: Initialized, Role: FOLLOWER
I20260809 16:33:44.566908 29914 consensus_queue.cc:260] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada4948894fb4a12a8203ba223ba46aa" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34145 } } peers { permanent_uuid: "d0ffccf80626424eba7f25baeeb03035" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 39803 } } peers { permanent_uuid: "d27549814fc94315b4aff0f038c991ef" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40623 } }
I20260809 16:33:44.566969 29921 tablet_bootstrap.cc:654] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:44.567190 29914 ts_tablet_manager.cc:1428] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 16:33:44.567289 29921 log.cc:824] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:44.567337 29893 heartbeater.cc:503] Master 127.30.23.190:34147 was elected leader, sending a full tablet report...
I20260809 16:33:44.567392 29916 raft_consensus.cc:348] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada4948894fb4a12a8203ba223ba46aa" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34145 } } peers { permanent_uuid: "d0ffccf80626424eba7f25baeeb03035" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 39803 } } peers { permanent_uuid: "d27549814fc94315b4aff0f038c991ef" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40623 } }
I20260809 16:33:44.567588 29916 raft_consensus.cc:374] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:44.567639 29916 raft_consensus.cc:729] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d27549814fc94315b4aff0f038c991ef, State: Initialized, Role: FOLLOWER
I20260809 16:33:44.567775 29916 consensus_queue.cc:260] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada4948894fb4a12a8203ba223ba46aa" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34145 } } peers { permanent_uuid: "d0ffccf80626424eba7f25baeeb03035" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 39803 } } peers { permanent_uuid: "d27549814fc94315b4aff0f038c991ef" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40623 } }
I20260809 16:33:44.568063 29916 ts_tablet_manager.cc:1428] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:33:44.568106 29921 tablet_bootstrap.cc:492] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035: No bootstrap required, opened a new log
I20260809 16:33:44.568140 29665 heartbeater.cc:503] Master 127.30.23.190:34147 was elected leader, sending a full tablet report...
I20260809 16:33:44.568188 29921 ts_tablet_manager.cc:1397] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:33:44.570695 29921 raft_consensus.cc:348] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada4948894fb4a12a8203ba223ba46aa" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34145 } } peers { permanent_uuid: "d0ffccf80626424eba7f25baeeb03035" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 39803 } } peers { permanent_uuid: "d27549814fc94315b4aff0f038c991ef" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40623 } }
I20260809 16:33:44.570851 29921 raft_consensus.cc:374] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:44.570875 29921 raft_consensus.cc:729] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0ffccf80626424eba7f25baeeb03035, State: Initialized, Role: FOLLOWER
I20260809 16:33:44.571082 29921 consensus_queue.cc:260] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada4948894fb4a12a8203ba223ba46aa" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34145 } } peers { permanent_uuid: "d0ffccf80626424eba7f25baeeb03035" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 39803 } } peers { permanent_uuid: "d27549814fc94315b4aff0f038c991ef" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40623 } }
I20260809 16:33:44.571362 29921 ts_tablet_manager.cc:1428] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:33:44.571439 29449 heartbeater.cc:503] Master 127.30.23.190:34147 was elected leader, sending a full tablet report...
W20260809 16:33:44.622834 29666 tablet.cc:2369] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:33:44.676772 29922 raft_consensus.cc:482] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:33:44.676990 29922 raft_consensus.cc:504] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada4948894fb4a12a8203ba223ba46aa" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34145 } } peers { permanent_uuid: "d0ffccf80626424eba7f25baeeb03035" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 39803 } } peers { permanent_uuid: "d27549814fc94315b4aff0f038c991ef" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40623 } }
I20260809 16:33:44.677497 29922 leader_election.cc:288] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0ffccf80626424eba7f25baeeb03035 (127.30.23.129:39803), d27549814fc94315b4aff0f038c991ef (127.30.23.130:40623)
I20260809 16:33:44.681694 29912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.677902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:44535 (local address 127.30.23.129:39803)
0809 16:33:44.677959 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:44.677962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:44.677977 (+ 15us) server_negotiation.cc:407] Connection header received
0809 16:33:44.678012 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:44.678013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:44.678027 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:44.678089 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:44.678277 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.679070 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.680643 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.681247 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.681334 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.681369 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:44.681566 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 16:33:44.681619 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:33:44.681702 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.677775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:39803 (local address 127.30.23.131:44535)
0809 16:33:44.677840 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:44.677853 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:44.677938 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:44.678101 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:44.678102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:44.678104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:44.678184 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.678186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.679200 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.679201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.680514 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.680517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.681384 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.681385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.681470 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.681474 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:44.681545 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 16:33:44.681599 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":31}
I20260809 16:33:44.681713 29930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.677903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:40623 (local address 127.30.23.131:55735)
0809 16:33:44.678078 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:44.678088 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:44.678135 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:44.678325 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:44.678326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:44.678327 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:44.678378 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.678379 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.679207 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.679208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.680514 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:44.680517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.681409 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.681409 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:44.681506 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.681509 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:33:44.681562 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 16:33:44.681599 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":39,"thread_start_us":117,"threads_started":1}
I20260809 16:33:44.681792 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.678030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:55735 (local address 127.30.23.130:40623)
0809 16:33:44.678096 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:44.678098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:44.678127 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:33:44.678205 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:44.678206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:44.678216 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:44.678300 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:44.678449 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.679119 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.680646 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.681247 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.681334 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.681370 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:44.681646 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 16:33:44.681721 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:33:44.682152 29586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adcf75f230a74b2a82c1f3d98b160df7" candidate_uuid: "ada4948894fb4a12a8203ba223ba46aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d27549814fc94315b4aff0f038c991ef" is_pre_election: true
I20260809 16:33:44.682150 29370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adcf75f230a74b2a82c1f3d98b160df7" candidate_uuid: "ada4948894fb4a12a8203ba223ba46aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0ffccf80626424eba7f25baeeb03035" is_pre_election: true
I20260809 16:33:44.682325 29370 raft_consensus.cc:2454] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ada4948894fb4a12a8203ba223ba46aa in term 0.
I20260809 16:33:44.682325 29586 raft_consensus.cc:2454] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ada4948894fb4a12a8203ba223ba46aa in term 0.
I20260809 16:33:44.682574 29711 leader_election.cc:302] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ada4948894fb4a12a8203ba223ba46aa, d27549814fc94315b4aff0f038c991ef; no voters:
I20260809 16:33:44.682708 29922 raft_consensus.cc:2781] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:33:44.682765 29922 raft_consensus.cc:482] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:33:44.682807 29922 raft_consensus.cc:3037] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:44.683543 29922 raft_consensus.cc:504] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada4948894fb4a12a8203ba223ba46aa" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34145 } } peers { permanent_uuid: "d0ffccf80626424eba7f25baeeb03035" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 39803 } } peers { permanent_uuid: "d27549814fc94315b4aff0f038c991ef" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40623 } }
I20260809 16:33:44.683683 29922 leader_election.cc:288] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [CANDIDATE]: Term 1 election: Requested vote from peers d0ffccf80626424eba7f25baeeb03035 (127.30.23.129:39803), d27549814fc94315b4aff0f038c991ef (127.30.23.130:40623)
I20260809 16:33:44.683888 29586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adcf75f230a74b2a82c1f3d98b160df7" candidate_uuid: "ada4948894fb4a12a8203ba223ba46aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d27549814fc94315b4aff0f038c991ef"
I20260809 16:33:44.683900 29370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adcf75f230a74b2a82c1f3d98b160df7" candidate_uuid: "ada4948894fb4a12a8203ba223ba46aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0ffccf80626424eba7f25baeeb03035"
I20260809 16:33:44.683982 29370 raft_consensus.cc:3037] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:44.683982 29586 raft_consensus.cc:3037] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:44.684922 29586 raft_consensus.cc:2454] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada4948894fb4a12a8203ba223ba46aa in term 1.
I20260809 16:33:44.684922 29370 raft_consensus.cc:2454] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada4948894fb4a12a8203ba223ba46aa in term 1.
I20260809 16:33:44.685153 29711 leader_election.cc:302] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ada4948894fb4a12a8203ba223ba46aa, d27549814fc94315b4aff0f038c991ef; no voters:
I20260809 16:33:44.685254 29922 raft_consensus.cc:2781] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:44.685465 29922 raft_consensus.cc:686] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [term 1 LEADER]: Becoming Leader. State: Replica: ada4948894fb4a12a8203ba223ba46aa, State: Running, Role: LEADER
I20260809 16:33:44.685549 29922 consensus_queue.cc:237] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada4948894fb4a12a8203ba223ba46aa" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34145 } } peers { permanent_uuid: "d0ffccf80626424eba7f25baeeb03035" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 39803 } } peers { permanent_uuid: "d27549814fc94315b4aff0f038c991ef" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40623 } }
I20260809 16:33:44.686522 29153 catalog_manager.cc:5495] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa reported cstate change: term changed from 0 to 1, leader changed from <none> to ada4948894fb4a12a8203ba223ba46aa (127.30.23.131). New cstate: current_term: 1 leader_uuid: "ada4948894fb4a12a8203ba223ba46aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada4948894fb4a12a8203ba223ba46aa" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 34145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0ffccf80626424eba7f25baeeb03035" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 39803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d27549814fc94315b4aff0f038c991ef" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40623 } health_report { overall_health: UNKNOWN } } }
I20260809 16:33:44.704684 29910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.702091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41336 (local address 127.30.23.131:34145)
0809 16:33:44.702159 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:44.702161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:44.702172 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:33:44.702284 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:44.702285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:44.702309 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:44.702388 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:33:44.702565 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.703074 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.703865 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:44.704080 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:44.704157 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:44.704311 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:33:44.704419 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:33:44.704530 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:44.704561 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 16:33:44.704570 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:33:44.706773 29370 raft_consensus.cc:1261] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035 [term 1 FOLLOWER]: Refusing update from remote peer ada4948894fb4a12a8203ba223ba46aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:44.706773 29586 raft_consensus.cc:1261] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef [term 1 FOLLOWER]: Refusing update from remote peer ada4948894fb4a12a8203ba223ba46aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:44.707198 29932 consensus_queue.cc:1035] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0ffccf80626424eba7f25baeeb03035" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 39803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:44.707376 29922 consensus_queue.cc:1035] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d27549814fc94315b4aff0f038c991ef" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:44.710070 29936 mvcc.cc:205] Tried to move back new op lower bound from 7316657048395866112 to 7316657048312741888. Current Snapshot: MvccSnapshot[applied={T|T < 7316657048395866112}]
I20260809 16:33:44.710850 29937 mvcc.cc:205] Tried to move back new op lower bound from 7316657048395866112 to 7316657048312741888. Current Snapshot: MvccSnapshot[applied={T|T < 7316657048395866112}]
I20260809 16:33:44.713057 29153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:44.713168 29153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:44.714244 29153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5eaee9158164adf83bd89fa9ac9c47d: Sending DeleteTablet for 3 replicas of tablet adcf75f230a74b2a82c1f3d98b160df7
I20260809 16:33:44.714591 29338 tablet_service.cc:1514] Processing DeleteTablet for tablet adcf75f230a74b2a82c1f3d98b160df7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:44 UTC) from {username='slave'} at 127.0.0.1:34562
I20260809 16:33:44.714748 29550 tablet_service.cc:1514] Processing DeleteTablet for tablet adcf75f230a74b2a82c1f3d98b160df7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:44 UTC) from {username='slave'} at 127.0.0.1:45176
I20260809 16:33:44.714900 29950 tablet_replica.cc:330] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035: stopping tablet replica
I20260809 16:33:44.714974 29784 tablet_service.cc:1514] Processing DeleteTablet for tablet adcf75f230a74b2a82c1f3d98b160df7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:44 UTC) from {username='slave'} at 127.0.0.1:41320
I20260809 16:33:44.715027 29950 raft_consensus.cc:2229] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:44.715149 29950 raft_consensus.cc:2258] T adcf75f230a74b2a82c1f3d98b160df7 P d0ffccf80626424eba7f25baeeb03035 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:44.715291 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 29234
I20260809 16:33:44.715390 29955 tablet_replica.cc:330] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa: stopping tablet replica
I20260809 16:33:44.715523 29955 raft_consensus.cc:2229] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:33:44.715705 29955 raft_consensus.cc:2258] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:44.718823 29955 ts_tablet_manager.cc:1905] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:44.721231 29955 ts_tablet_manager.cc:1918] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:44.721444 29955 log.cc:1196] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/adcf75f230a74b2a82c1f3d98b160df7
I20260809 16:33:44.721885 29955 ts_tablet_manager.cc:1939] T adcf75f230a74b2a82c1f3d98b160df7 P ada4948894fb4a12a8203ba223ba46aa: Deleting consensus metadata
I20260809 16:33:44.722543 29133 catalog_manager.cc:4841] TS ada4948894fb4a12a8203ba223ba46aa (127.30.23.131:34145): tablet adcf75f230a74b2a82c1f3d98b160df7 (table test-table [id=aefc8dd385da4d5eae22e86dd8ec3997]) successfully deleted
I20260809 16:33:44.731009 29952 tablet_replica.cc:330] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef: stopping tablet replica
I20260809 16:33:44.731189 29952 raft_consensus.cc:2229] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:44.731335 29952 raft_consensus.cc:2258] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:44.733680 29952 ts_tablet_manager.cc:1905] T adcf75f230a74b2a82c1f3d98b160df7 P d27549814fc94315b4aff0f038c991ef: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 16:33:44.735646 29127 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:39803 (error 108)
I20260809 16:33:44.735942 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 29453
I20260809 16:33:44.736239 29909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.736101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:39803 (local address 127.0.0.1:34570)
0809 16:33:44.736160 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:44.736201 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:39803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 16:33:44.772365 29127 catalog_manager.cc:4584] TS d0ffccf80626424eba7f25baeeb03035 (127.30.23.129:39803): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet adcf75f230a74b2a82c1f3d98b160df7: Network error: Client connection negotiation failed: client connection to 127.30.23.129:39803: connect: Connection refused (error 111)
I20260809 16:33:44.777935 29909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:44.777758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:40623 (local address 127.0.0.1:45180)
0809 16:33:44.777835 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:44.777877 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.130:40623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 16:33:44.778041 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 29669
I20260809 16:33:44.792357 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 29105
2026-08-09T16:33:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (940 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:33:44 dist-test-slave-j2vl krb5kdc[29966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:33:44 dist-test-slave-j2vl krb5kdc[29966](info): set up 2 sockets
Aug 09 16:33:44 dist-test-slave-j2vl krb5kdc[29966](info): commencing operation
krb5kdc: starting...
W20260809 16:33:46.145081 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:33:46 dist-test-slave-j2vl krb5kdc[29966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:33:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:33:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:33:46.290398 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:43181
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:40801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:43181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:46.383261 29988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:46.383518 29988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:46.383545 29988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:46.384720 29988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:46.384760 29988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:46.384774 29988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:46.384788 29988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:46.384800 29988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:46.386420 29988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:43181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:43181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:46.386711 29988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:46.387001 29988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:46.392200 29994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:46.392225 29995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:46.392200 29997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:46.392499 29988 server_base.cc:1034] running on GCE node
I20260809 16:33:46.392771 29988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:46.393095 29988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:46.394251 29988 hybrid_clock.cc:648] HybridClock initialized: now 1786293226394229 us; error 35 us; skew 500 ppm
Aug 09 16:33:46 dist-test-slave-j2vl krb5kdc[29966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293226, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:46.396938 29988 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:33:46.397523 29988 webserver.cc:460] Webserver started at http://127.30.23.190:44957/ using document root <none> and password file <none>
I20260809 16:33:46.397773 29988 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:46.397850 29988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:46.398018 29988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:46.399173 29988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "c68f5bcfcc8b4a518c24f812af4b8bed"
format_stamp: "Formatted at 2026-08-09 16:33:46 on dist-test-slave-j2vl"
server_key: "402b1441045bf50cb0a4329cb79a7930"
server_key_iv: "7715fd43b3e7d29e6f293fb0f06a216f"
server_key_version: "encryptionkey@0"
I20260809 16:33:46.399582 29988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "c68f5bcfcc8b4a518c24f812af4b8bed"
format_stamp: "Formatted at 2026-08-09 16:33:46 on dist-test-slave-j2vl"
server_key: "402b1441045bf50cb0a4329cb79a7930"
server_key_iv: "7715fd43b3e7d29e6f293fb0f06a216f"
server_key_version: "encryptionkey@0"
I20260809 16:33:46.401100 29988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:33:46.402156 30007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:46.402406 29988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:33:46.402523 29988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "c68f5bcfcc8b4a518c24f812af4b8bed"
format_stamp: "Formatted at 2026-08-09 16:33:46 on dist-test-slave-j2vl"
server_key: "402b1441045bf50cb0a4329cb79a7930"
server_key_iv: "7715fd43b3e7d29e6f293fb0f06a216f"
server_key_version: "encryptionkey@0"
I20260809 16:33:46.402609 29988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:46.416663 29988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:46.419623 29988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:46.419855 29988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:46.425151 29988 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:43181
I20260809 16:33:46.425184 30092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:43181 every 8 connection(s)
I20260809 16:33:46.425767 29988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:46.426507 30094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:46.427528 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 29988
I20260809 16:33:46.427675 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:33:46.427975 30814 external_mini_cluster.cc:1428] Setting key 6a013e6b2e71df269a8e18b69db0531a
I20260809 16:33:46.430311 30094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed: Bootstrap starting.
I20260809 16:33:46.431111 30094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:46.431445 30094 log.cc:824] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:46.432734 30094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed: No bootstrap required, opened a new log
I20260809 16:33:46.435316 30094 raft_consensus.cc:348] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68f5bcfcc8b4a518c24f812af4b8bed" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43181 } }
I20260809 16:33:46.435504 30094 raft_consensus.cc:374] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:46.435559 30094 raft_consensus.cc:729] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c68f5bcfcc8b4a518c24f812af4b8bed, State: Initialized, Role: FOLLOWER
I20260809 16:33:46.435746 30094 consensus_queue.cc:260] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68f5bcfcc8b4a518c24f812af4b8bed" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43181 } }
I20260809 16:33:46.435853 30094 raft_consensus.cc:388] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:46.435899 30094 raft_consensus.cc:482] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 09 16:33:46 dist-test-slave-j2vl krb5kdc[29966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:33:46.435950 30094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:46.436733 30094 raft_consensus.cc:504] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68f5bcfcc8b4a518c24f812af4b8bed" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43181 } }
I20260809 16:33:46.436854 30094 leader_election.cc:302] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c68f5bcfcc8b4a518c24f812af4b8bed; no voters:
I20260809 16:33:46.437047 30094 leader_election.cc:288] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:33:46.437156 30102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:46.437386 30102 raft_consensus.cc:686] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [term 1 LEADER]: Becoming Leader. State: Replica: c68f5bcfcc8b4a518c24f812af4b8bed, State: Running, Role: LEADER
I20260809 16:33:46.437410 30094 sys_catalog.cc:564] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:46.437814 30102 consensus_queue.cc:237] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68f5bcfcc8b4a518c24f812af4b8bed" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43181 } }
I20260809 16:33:46.438218 30105 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [sys.catalog]: SysCatalogTable state changed. Reason: New leader c68f5bcfcc8b4a518c24f812af4b8bed. Latest consensus state: current_term: 1 leader_uuid: "c68f5bcfcc8b4a518c24f812af4b8bed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68f5bcfcc8b4a518c24f812af4b8bed" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43181 } } }
I20260809 16:33:46.438297 30105 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:46.438800 30104 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c68f5bcfcc8b4a518c24f812af4b8bed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68f5bcfcc8b4a518c24f812af4b8bed" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 43181 } } }
I20260809 16:33:46.438937 30104 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:46.439306 30111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:46.440092 30111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:46.441076 30099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:46.428696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54936 (local address 127.30.23.190:43181)
0809 16:33:46.428905 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:46.428916 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:46.428957 (+ 41us) server_negotiation.cc:407] Connection header received
0809 16:33:46.429472 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:46.429479 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:46.429537 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:46.429696 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:46.430416 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:46.432008 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:46.432959 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:46.433614 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:46.433773 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:46.436425 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:46.436430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:46.436430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:46.436445 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:46.438786 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:46.439297 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:46.439300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:46.439301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:46.439485 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:46.439740 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:46.439741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:46.439742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:46.440094 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:46.440668 (+ 574us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:46.440749 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 16:33:46.440910 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
I20260809 16:33:46.442211 30111 catalog_manager.cc:1347] Generated new cluster ID: da59b44edd1642d78c4267c80eacc21e
I20260809 16:33:46.442276 30111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:33:46.455699 30111 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:33:46.456282 30111 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:46.466180 30111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed: Generated new TSK 0
I20260809 16:33:46.466476 30111 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.23.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
Entry for principal kudu/127.30.23.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab.
I20260809 16:33:46.509538 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43181
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:40801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:46.605422 30132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:46.605717 30132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:46.605831 30132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:46.607158 30132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:46.607234 30132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:46.607280 30132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:33:46.608949 30132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.129.keytab
--principal=kudu/127.30.23.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:46.609251 30132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:46.609556 30132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:46.615206 30142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:46.615198 30143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:46.615206 30145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:46.615224 30132 server_base.cc:1034] running on GCE node
I20260809 16:33:46.615770 30132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:46.615999 30132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:46.617169 30132 hybrid_clock.cc:648] HybridClock initialized: now 1786293226617142 us; error 41 us; skew 500 ppm
Aug 09 16:33:46 dist-test-slave-j2vl krb5kdc[29966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293226, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:46.620157 30132 init.cc:377] Logged in from keytab as kudu/127.30.23.129@KRBTEST.COM (short username kudu)
I20260809 16:33:46.620731 30132 webserver.cc:460] Webserver started at http://127.30.23.129:37679/ using document root <none> and password file <none>
I20260809 16:33:46.621012 30132 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:46.621061 30132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:46.621201 30132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:46.622395 30132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "add3a565be794b079118ae4cb85eb2d6"
format_stamp: "Formatted at 2026-08-09 16:33:46 on dist-test-slave-j2vl"
server_key: "3ad821bbb64534b5002263daab1fcb85"
server_key_iv: "24ee5a88c9f401b688e706a867d22410"
server_key_version: "encryptionkey@0"
I20260809 16:33:46.622784 30132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "add3a565be794b079118ae4cb85eb2d6"
format_stamp: "Formatted at 2026-08-09 16:33:46 on dist-test-slave-j2vl"
server_key: "3ad821bbb64534b5002263daab1fcb85"
server_key_iv: "24ee5a88c9f401b688e706a867d22410"
server_key_version: "encryptionkey@0"
I20260809 16:33:46.624394 30132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:33:46.625594 30155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:46.625844 30132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:33:46.625970 30132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "add3a565be794b079118ae4cb85eb2d6"
format_stamp: "Formatted at 2026-08-09 16:33:46 on dist-test-slave-j2vl"
server_key: "3ad821bbb64534b5002263daab1fcb85"
server_key_iv: "24ee5a88c9f401b688e706a867d22410"
server_key_version: "encryptionkey@0"
I20260809 16:33:46.626057 30132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:46.644032 30132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:46.646791 30132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:46.647004 30132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:46.647297 30132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:46.647682 30132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:46.647739 30132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:46.647809 30132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:46.647851 30132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:46.655303 30132 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:38045
I20260809 16:33:46.655362 30363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:38045 every 8 connection(s)
I20260809 16:33:46.655783 30132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:33:46.657133 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 30132
I20260809 16:33:46.657270 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:33:46.657511 30814 external_mini_cluster.cc:1428] Setting key 10f20b919c6f1e9f2a0849f08135e1af
Aug 09 16:33:46 dist-test-slave-j2vl krb5kdc[29966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293226, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.129@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.23.130@KRBTEST.COM; defaulting to no policy
I20260809 16:33:46.667339 30099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:46.656781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:51087 (local address 127.30.23.190:43181)
0809 16:33:46.656830 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:46.656832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:46.657042 (+ 210us) server_negotiation.cc:407] Connection header received
0809 16:33:46.657756 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:46.657757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:46.657772 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:46.657832 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:46.659009 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:46.659792 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:46.660941 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:46.661179 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:46.661298 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:46.663996 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:46.663998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:46.663999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:46.664009 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:46.665697 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:46.666184 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:46.666186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:46.666187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:46.666256 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:46.666527 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:46.666528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:46.666528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:46.666661 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:46.666744 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:46.667097 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 16:33:46.667213 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:33:46.667419 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:46.656748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:43181 (local address 127.30.23.129:51087)
0809 16:33:46.656913 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:46.656955 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:46.657613 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:46.657915 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:46.657921 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:46.658664 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:46.658902 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:46.658907 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:46.659918 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:46.659919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:46.660805 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:46.660807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:46.661281 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:46.661283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:46.661459 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:46.662049 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:46.662058 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:46.663846 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:46.665794 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:46.665805 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:46.665808 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:46.666098 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:46.666334 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:46.666335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:46.666335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:46.666455 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:46.666733 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:46.666741 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:46.666907 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 16:33:46.667038 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 16:33:46.667208 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260809 16:33:46.667959 30364 heartbeater.cc:347] Connected to a master server at 127.30.23.190:43181
I20260809 16:33:46.668077 30364 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:46.668277 30364 heartbeater.cc:511] Master 127.30.23.190:43181 requested a full tablet report, sending...
I20260809 16:33:46.668907 30035 ts_manager.cc:194] Registered new tserver with Master: add3a565be794b079118ae4cb85eb2d6 (127.30.23.129:38045)
I20260809 16:33:46.669646 30035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.129@KRBTEST.COM'} at 127.30.23.129:51087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
Entry for principal kudu/127.30.23.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab.
I20260809 16:33:46.689568 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43181
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:40801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:46.781508 30372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:46.781813 30372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:46.781848 30372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:46.783440 30372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:46.783509 30372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:46.783548 30372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:33:46.785485 30372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.130.keytab
--principal=kudu/127.30.23.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:46.785830 30372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:46.786114 30372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:46.794107 30384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:46.794107 30378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:46.794274 30380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:46.794144 30372 server_base.cc:1034] running on GCE node
I20260809 16:33:46.795449 30372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:46.796324 30372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:46.797607 30372 hybrid_clock.cc:648] HybridClock initialized: now 1786293226797578 us; error 52 us; skew 500 ppm
Aug 09 16:33:46 dist-test-slave-j2vl krb5kdc[29966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293226, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:46.801461 30372 init.cc:377] Logged in from keytab as kudu/127.30.23.130@KRBTEST.COM (short username kudu)
I20260809 16:33:46.802089 30372 webserver.cc:460] Webserver started at http://127.30.23.130:42103/ using document root <none> and password file <none>
I20260809 16:33:46.802399 30372 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:46.802487 30372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:46.802662 30372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:46.803938 30372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "6467aa115f2046b582c96b62cd598453"
format_stamp: "Formatted at 2026-08-09 16:33:46 on dist-test-slave-j2vl"
server_key: "c6e35b2d7f1f210e4e58685e4816ccfa"
server_key_iv: "a0d06f39e766ce55e7ea9c817e4a344b"
server_key_version: "encryptionkey@0"
I20260809 16:33:46.804385 30372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "6467aa115f2046b582c96b62cd598453"
format_stamp: "Formatted at 2026-08-09 16:33:46 on dist-test-slave-j2vl"
server_key: "c6e35b2d7f1f210e4e58685e4816ccfa"
server_key_iv: "a0d06f39e766ce55e7ea9c817e4a344b"
server_key_version: "encryptionkey@0"
I20260809 16:33:46.806851 30372 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 16:33:46.808056 30398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:46.808447 30372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:46.808589 30372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "6467aa115f2046b582c96b62cd598453"
format_stamp: "Formatted at 2026-08-09 16:33:46 on dist-test-slave-j2vl"
server_key: "c6e35b2d7f1f210e4e58685e4816ccfa"
server_key_iv: "a0d06f39e766ce55e7ea9c817e4a344b"
server_key_version: "encryptionkey@0"
I20260809 16:33:46.808688 30372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:46.846669 30372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:46.850062 30372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:46.850270 30372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:46.850556 30372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:46.850920 30372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:46.851315 30372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:46.851373 30372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:46.851404 30372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:46.859599 30372 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:40485
I20260809 16:33:46.859644 30578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:40485 every 8 connection(s)
I20260809 16:33:46.860157 30372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:33:46 dist-test-slave-j2vl krb5kdc[29966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293226, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.130@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:33:46.868367 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 30372
I20260809 16:33:46.868537 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:33:46.868829 30814 external_mini_cluster.cc:1428] Setting key ecc9710755350b2464724274623ce6d0
I20260809 16:33:46.872253 30099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:46.861180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:36053 (local address 127.30.23.190:43181)
0809 16:33:46.861244 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:46.861246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:46.861498 (+ 252us) server_negotiation.cc:407] Connection header received
0809 16:33:46.862011 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:46.862013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:46.862025 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:46.862095 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:46.863308 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:46.863842 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:46.864971 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:46.865168 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:46.865239 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:46.868153 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:46.868157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:46.868157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:46.868172 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:46.870371 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:46.870934 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:46.870936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:46.870936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:46.871012 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:46.871332 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:46.871334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:46.871334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:46.871478 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:46.871694 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:46.872051 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 16:33:46.872155 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:33:46.872484 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:46.861014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:43181 (local address 127.30.23.130:36053)
0809 16:33:46.861203 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:46.861249 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:46.861864 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:46.862237 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:46.862242 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:46.862946 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:46.863172 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:46.863177 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:46.863978 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:46.863980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:46.864839 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:46.864842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:46.865276 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:46.865277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:46.865532 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:46.866190 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:46.866200 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:46.867961 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:46.870490 (+ 2529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:46.870503 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:46.870507 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:46.870831 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:46.871114 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:46.871115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:46.871116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:46.871251 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:46.871679 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:46.871688 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:46.871875 (+ 187us) client_negotiation.cc:769] Sending connection context
0809 16:33:46.872011 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 16:33:46.872201 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
I20260809 16:33:46.873066 30579 heartbeater.cc:347] Connected to a master server at 127.30.23.190:43181
I20260809 16:33:46.873172 30579 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:46.873461 30579 heartbeater.cc:511] Master 127.30.23.190:43181 requested a full tablet report, sending...
I20260809 16:33:46.873904 30035 ts_manager.cc:194] Registered new tserver with Master: 6467aa115f2046b582c96b62cd598453 (127.30.23.130:40485)
I20260809 16:33:46.874320 30035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.130@KRBTEST.COM'} at 127.30.23.130:36053
WARNING: no policy specified for kudu/127.30.23.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
Entry for principal kudu/127.30.23.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab.
I20260809 16:33:46.901408 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43181
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:40801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:46.995107 30586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:46.995394 30586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:46.995440 30586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:46.996699 30586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:46.996943 30586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:46.997044 30586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:33:46.998821 30586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:40801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.131.keytab
--principal=kudu/127.30.23.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:43181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:46.999148 30586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:46.999648 30586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:47.006618 30597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:47.006613 30600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:47.006613 30604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:47.006968 30586 server_base.cc:1034] running on GCE node
I20260809 16:33:47.007221 30586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:47.007522 30586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:47.008723 30586 hybrid_clock.cc:648] HybridClock initialized: now 1786293227008705 us; error 34 us; skew 500 ppm
Aug 09 16:33:47 dist-test-slave-j2vl krb5kdc[29966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293227, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:47.012079 30586 init.cc:377] Logged in from keytab as kudu/127.30.23.131@KRBTEST.COM (short username kudu)
I20260809 16:33:47.012655 30586 webserver.cc:460] Webserver started at http://127.30.23.131:41629/ using document root <none> and password file <none>
I20260809 16:33:47.012948 30586 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:47.013001 30586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:47.013159 30586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:47.014526 30586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "3a802b1bf0ea4629a7abbf5880eaf775"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "3876fad324dfcbadaf266080c066df88"
server_key_iv: "aa56198c6d8d844a4234725ad1147345"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.015056 30586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "3a802b1bf0ea4629a7abbf5880eaf775"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "3876fad324dfcbadaf266080c066df88"
server_key_iv: "aa56198c6d8d844a4234725ad1147345"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.017227 30586 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 16:33:47.018325 30616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:47.018651 30586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:47.018770 30586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "3a802b1bf0ea4629a7abbf5880eaf775"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "3876fad324dfcbadaf266080c066df88"
server_key_iv: "aa56198c6d8d844a4234725ad1147345"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.018874 30586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:47.031816 30586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:47.034657 30586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:47.034896 30586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:47.035182 30586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:47.035555 30586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:47.035619 30586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:47.035667 30586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:47.035696 30586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:47.042897 30586 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:43157
I20260809 16:33:47.042963 30804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:43157 every 8 connection(s)
I20260809 16:33:47.043339 30586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:33:47.048475 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 30586
I20260809 16:33:47.048586 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:33:47.048812 30814 external_mini_cluster.cc:1428] Setting key 125cd0f90ef5e187850c4aaaea4cf5a2
Aug 09 16:33:47 dist-test-slave-j2vl krb5kdc[29966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293227, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.131@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:33:47.054188 30099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.044179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:45935 (local address 127.30.23.190:43181)
0809 16:33:47.044269 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.044271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.044528 (+ 257us) server_negotiation.cc:407] Connection header received
0809 16:33:47.045097 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.045099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.045112 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.045168 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:47.046245 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.046766 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.047792 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.048001 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.048085 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.050938 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:47.050940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:47.050940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:47.050949 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:47.052421 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:47.052915 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:47.052917 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:47.052917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:47.052983 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:47.053276 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:47.053277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:47.053278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:47.053480 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:47.053556 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.053902 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.054076 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:33:47.054450 30811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.044151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:43181 (local address 127.30.23.131:45935)
0809 16:33:47.044342 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:47.044411 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:47.044978 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:47.045237 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:47.045248 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:47.045939 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:47.046138 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.046144 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.046884 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.046885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.047693 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.047696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.048094 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.048096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.048265 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.048988 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:33:47.049000 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:47.050826 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:47.052562 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:47.052572 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:47.052574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:47.052831 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:47.053057 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:33:47.053058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:33:47.053058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:33:47.053196 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:33:47.053561 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:47.053569 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:47.053725 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 16:33:47.053878 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 16:33:47.054180 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260809 16:33:47.055140 30808 heartbeater.cc:347] Connected to a master server at 127.30.23.190:43181
I20260809 16:33:47.055260 30808 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:47.055449 30808 heartbeater.cc:511] Master 127.30.23.190:43181 requested a full tablet report, sending...
I20260809 16:33:47.055938 30035 ts_manager.cc:194] Registered new tserver with Master: 3a802b1bf0ea4629a7abbf5880eaf775 (127.30.23.131:43157)
I20260809 16:33:47.056344 30035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.23.131@KRBTEST.COM'} at 127.30.23.131:45935
I20260809 16:33:47.061733 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:33:47.070672 30099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.063326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.30.23.190:43181)
0809 16:33:47.063410 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.063412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.063580 (+ 168us) server_negotiation.cc:407] Connection header received
0809 16:33:47.063614 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.063615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.063626 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.063686 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:47.064411 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.065017 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.066079 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.066273 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.066339 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.067770 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:47.067778 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:47.067778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:47.067786 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:47.069271 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:47.069737 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:47.069738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:47.069738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:47.069802 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:47.070051 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:47.070053 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:47.070053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:47.070244 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:47.070347 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.070509 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.070588 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:33:47.072495 30035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:33:47.074780 30035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:33:47.082331 30833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.077542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42964 (local address 127.30.23.130:40485)
0809 16:33:47.077896 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.077901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.077913 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:33:47.078007 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.078010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.078030 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.078089 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:47.078474 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.079673 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.081023 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.081831 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.081919 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.081988 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.082179 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.082232 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":98,"threads_started":1}
I20260809 16:33:47.082316 30834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.077701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:38045 (local address 127.0.0.1:54576)
0809 16:33:47.077994 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:47.078004 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:47.078053 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:47.078534 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:47.078535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:47.078537 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:47.078581 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.078583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.079896 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.079897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.081094 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.081097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.082022 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.082023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.082107 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.082112 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:33:47.082147 (+ 35us) client_negotiation.cc:240] Negotiation successful
0809 16:33:47.082220 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":30,"thread_start_us":250,"threads_started":1}
I20260809 16:33:47.082406 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.078172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40036 (local address 127.30.23.131:43157)
0809 16:33:47.078374 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.078378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.078389 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:33:47.078431 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.078433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.078451 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.078700 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:47.078751 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.079590 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.081430 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.082039 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.082122 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.082187 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.082272 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.082311 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":94,"threads_started":1}
I20260809 16:33:47.082626 30832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.077461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:40485 (local address 127.0.0.1:42964)
0809 16:33:47.077608 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:47.077656 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:47.077739 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:47.078288 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:47.078291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:47.078294 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:47.078376 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.078378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.079806 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.079807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.080925 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.080928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.081942 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.081943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.082036 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.082053 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 16:33:47.082491 (+ 438us) client_negotiation.cc:240] Negotiation successful
0809 16:33:47.082546 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":53,"thread_start_us":88,"threads_started":1}
I20260809 16:33:47.082860 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.077863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54576 (local address 127.30.23.129:38045)
0809 16:33:47.078205 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.078209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.078219 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:33:47.078408 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.078410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.078432 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.078653 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:47.078723 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.079789 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.081191 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.081923 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.082622 (+ 699us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.082695 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.082723 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.082768 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":88,"threads_started":1}
I20260809 16:33:47.083427 30474 tablet_service.cc:1467] Processing CreateTablet for tablet dbfc7ef560e04adcb3d404fe387ce5c7 (DEFAULT_TABLE table=test-table [id=b1be46ebfe1645009fca22b12c067d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:47.083804 30474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbfc7ef560e04adcb3d404fe387ce5c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:47.085629 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.078064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:43157 (local address 127.0.0.1:40036)
0809 16:33:47.078199 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:47.078210 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:47.078263 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:47.078553 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:47.078554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:47.078556 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:47.078626 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.078627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.079732 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.079733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.081094 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.081097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.082132 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.082133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.082195 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.082198 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:33:47.082258 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:33:47.082298 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":6,"thread_start_us":81,"threads_started":1}
I20260809 16:33:47.086000 30236 tablet_service.cc:1467] Processing CreateTablet for tablet dbfc7ef560e04adcb3d404fe387ce5c7 (DEFAULT_TABLE table=test-table [id=b1be46ebfe1645009fca22b12c067d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:47.086407 30236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbfc7ef560e04adcb3d404fe387ce5c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:47.086455 30695 tablet_service.cc:1467] Processing CreateTablet for tablet dbfc7ef560e04adcb3d404fe387ce5c7 (DEFAULT_TABLE table=test-table [id=b1be46ebfe1645009fca22b12c067d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:47.087004 30695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbfc7ef560e04adcb3d404fe387ce5c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:47.087590 30840 tablet_bootstrap.cc:492] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453: Bootstrap starting.
I20260809 16:33:47.088424 30840 tablet_bootstrap.cc:654] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:47.089064 30840 log.cc:824] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:47.089985 30840 tablet_bootstrap.cc:492] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453: No bootstrap required, opened a new log
I20260809 16:33:47.090066 30840 ts_tablet_manager.cc:1397] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:33:47.091207 30846 tablet_bootstrap.cc:492] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775: Bootstrap starting.
I20260809 16:33:47.091940 30840 raft_consensus.cc:348] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6467aa115f2046b582c96b62cd598453" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40485 } } peers { permanent_uuid: "add3a565be794b079118ae4cb85eb2d6" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 38045 } } peers { permanent_uuid: "3a802b1bf0ea4629a7abbf5880eaf775" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 43157 } }
I20260809 16:33:47.092097 30840 raft_consensus.cc:374] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:47.092120 30840 raft_consensus.cc:729] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6467aa115f2046b582c96b62cd598453, State: Initialized, Role: FOLLOWER
I20260809 16:33:47.092259 30840 consensus_queue.cc:260] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6467aa115f2046b582c96b62cd598453" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40485 } } peers { permanent_uuid: "add3a565be794b079118ae4cb85eb2d6" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 38045 } } peers { permanent_uuid: "3a802b1bf0ea4629a7abbf5880eaf775" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 43157 } }
I20260809 16:33:47.092569 30846 tablet_bootstrap.cc:654] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:47.092584 30840 ts_tablet_manager.cc:1428] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:33:47.092736 30579 heartbeater.cc:503] Master 127.30.23.190:43181 was elected leader, sending a full tablet report...
I20260809 16:33:47.092767 30844 tablet_bootstrap.cc:492] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6: Bootstrap starting.
I20260809 16:33:47.093217 30846 log.cc:824] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:47.093593 30844 tablet_bootstrap.cc:654] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:47.094237 30846 tablet_bootstrap.cc:492] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775: No bootstrap required, opened a new log
I20260809 16:33:47.094218 30844 log.cc:824] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:47.094331 30846 ts_tablet_manager.cc:1397] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:33:47.095129 30844 tablet_bootstrap.cc:492] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6: No bootstrap required, opened a new log
I20260809 16:33:47.095212 30844 ts_tablet_manager.cc:1397] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260809 16:33:47.096457 30846 raft_consensus.cc:348] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6467aa115f2046b582c96b62cd598453" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40485 } } peers { permanent_uuid: "add3a565be794b079118ae4cb85eb2d6" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 38045 } } peers { permanent_uuid: "3a802b1bf0ea4629a7abbf5880eaf775" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 43157 } }
I20260809 16:33:47.096696 30846 raft_consensus.cc:374] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:47.096740 30846 raft_consensus.cc:729] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a802b1bf0ea4629a7abbf5880eaf775, State: Initialized, Role: FOLLOWER
I20260809 16:33:47.096892 30846 consensus_queue.cc:260] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6467aa115f2046b582c96b62cd598453" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40485 } } peers { permanent_uuid: "add3a565be794b079118ae4cb85eb2d6" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 38045 } } peers { permanent_uuid: "3a802b1bf0ea4629a7abbf5880eaf775" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 43157 } }
I20260809 16:33:47.097296 30846 ts_tablet_manager.cc:1428] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:33:47.097481 30808 heartbeater.cc:503] Master 127.30.23.190:43181 was elected leader, sending a full tablet report...
I20260809 16:33:47.097397 30844 raft_consensus.cc:348] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6467aa115f2046b582c96b62cd598453" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40485 } } peers { permanent_uuid: "add3a565be794b079118ae4cb85eb2d6" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 38045 } } peers { permanent_uuid: "3a802b1bf0ea4629a7abbf5880eaf775" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 43157 } }
I20260809 16:33:47.097584 30844 raft_consensus.cc:374] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:47.097627 30844 raft_consensus.cc:729] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: add3a565be794b079118ae4cb85eb2d6, State: Initialized, Role: FOLLOWER
I20260809 16:33:47.097769 30844 consensus_queue.cc:260] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6467aa115f2046b582c96b62cd598453" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40485 } } peers { permanent_uuid: "add3a565be794b079118ae4cb85eb2d6" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 38045 } } peers { permanent_uuid: "3a802b1bf0ea4629a7abbf5880eaf775" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 43157 } }
I20260809 16:33:47.098289 30844 ts_tablet_manager.cc:1428] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:33:47.098344 30364 heartbeater.cc:503] Master 127.30.23.190:43181 was elected leader, sending a full tablet report...
W20260809 16:33:47.110874 30581 tablet.cc:2369] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:33:47.156745 30365 tablet.cc:2369] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:33:47.249780 30853 raft_consensus.cc:482] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:33:47.249990 30853 raft_consensus.cc:504] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6467aa115f2046b582c96b62cd598453" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40485 } } peers { permanent_uuid: "add3a565be794b079118ae4cb85eb2d6" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 38045 } } peers { permanent_uuid: "3a802b1bf0ea4629a7abbf5880eaf775" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 43157 } }
I20260809 16:33:47.250804 30853 leader_election.cc:288] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6467aa115f2046b582c96b62cd598453 (127.30.23.130:40485), 3a802b1bf0ea4629a7abbf5880eaf775 (127.30.23.131:43157)
I20260809 16:33:47.254837 30857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.251298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:43157 (local address 127.30.23.129:45879)
0809 16:33:47.251478 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:47.251495 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:47.251573 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:47.252063 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:47.252065 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:47.252070 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:47.252253 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.252258 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.253024 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.253024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.253933 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.253936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.254570 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.254571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.254660 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.254664 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:47.254723 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:33:47.254756 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":60,"threads_started":1}
I20260809 16:33:47.254967 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.251557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:45879 (local address 127.30.23.131:43157)
0809 16:33:47.251820 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.251822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.251831 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:33:47.251857 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.251858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.251870 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.251948 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:47.252373 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.252914 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.254051 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.254462 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.254532 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.254541 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.254844 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.254896 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260809 16:33:47.255038 30833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.251216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:52999 (local address 127.30.23.130:40485)
0809 16:33:47.251507 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.251510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.251522 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:33:47.251556 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.251557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.251573 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.251644 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:47.252053 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.252716 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.253925 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.254491 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.254603 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.254612 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.254905 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.254955 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260}
I20260809 16:33:47.254997 30856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.251020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:40485 (local address 127.30.23.129:52999)
0809 16:33:47.251207 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:47.251220 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:47.251283 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:47.251820 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:47.251821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:47.251825 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:47.251931 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.251934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.252822 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.252824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.253797 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.253800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.254627 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.254628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.254685 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.254689 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:47.254875 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 16:33:47.254916 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":51,"thread_start_us":135,"threads_started":1}
I20260809 16:33:47.255491 30510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbfc7ef560e04adcb3d404fe387ce5c7" candidate_uuid: "add3a565be794b079118ae4cb85eb2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6467aa115f2046b582c96b62cd598453" is_pre_election: true
I20260809 16:33:47.255517 30724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbfc7ef560e04adcb3d404fe387ce5c7" candidate_uuid: "add3a565be794b079118ae4cb85eb2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a802b1bf0ea4629a7abbf5880eaf775" is_pre_election: true
I20260809 16:33:47.255638 30510 raft_consensus.cc:2454] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate add3a565be794b079118ae4cb85eb2d6 in term 0.
I20260809 16:33:47.255728 30724 raft_consensus.cc:2454] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate add3a565be794b079118ae4cb85eb2d6 in term 0.
I20260809 16:33:47.255924 30157 leader_election.cc:302] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6467aa115f2046b582c96b62cd598453, add3a565be794b079118ae4cb85eb2d6; no voters:
I20260809 16:33:47.256145 30853 raft_consensus.cc:2781] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:33:47.256210 30853 raft_consensus.cc:482] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:33:47.256240 30853 raft_consensus.cc:3037] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:47.257040 30853 raft_consensus.cc:504] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6467aa115f2046b582c96b62cd598453" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40485 } } peers { permanent_uuid: "add3a565be794b079118ae4cb85eb2d6" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 38045 } } peers { permanent_uuid: "3a802b1bf0ea4629a7abbf5880eaf775" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 43157 } }
I20260809 16:33:47.257218 30853 leader_election.cc:288] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [CANDIDATE]: Term 1 election: Requested vote from peers 6467aa115f2046b582c96b62cd598453 (127.30.23.130:40485), 3a802b1bf0ea4629a7abbf5880eaf775 (127.30.23.131:43157)
I20260809 16:33:47.257575 30724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbfc7ef560e04adcb3d404fe387ce5c7" candidate_uuid: "add3a565be794b079118ae4cb85eb2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a802b1bf0ea4629a7abbf5880eaf775"
I20260809 16:33:47.257673 30724 raft_consensus.cc:3037] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:47.257719 30510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbfc7ef560e04adcb3d404fe387ce5c7" candidate_uuid: "add3a565be794b079118ae4cb85eb2d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6467aa115f2046b582c96b62cd598453"
I20260809 16:33:47.257824 30510 raft_consensus.cc:3037] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:47.258592 30510 raft_consensus.cc:2454] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate add3a565be794b079118ae4cb85eb2d6 in term 1.
I20260809 16:33:47.258868 30157 leader_election.cc:302] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6467aa115f2046b582c96b62cd598453, add3a565be794b079118ae4cb85eb2d6; no voters:
I20260809 16:33:47.259016 30724 raft_consensus.cc:2454] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate add3a565be794b079118ae4cb85eb2d6 in term 1.
I20260809 16:33:47.259037 30853 raft_consensus.cc:2781] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:47.259255 30853 raft_consensus.cc:686] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [term 1 LEADER]: Becoming Leader. State: Replica: add3a565be794b079118ae4cb85eb2d6, State: Running, Role: LEADER
I20260809 16:33:47.259374 30853 consensus_queue.cc:237] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6467aa115f2046b582c96b62cd598453" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40485 } } peers { permanent_uuid: "add3a565be794b079118ae4cb85eb2d6" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 38045 } } peers { permanent_uuid: "3a802b1bf0ea4629a7abbf5880eaf775" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 43157 } }
I20260809 16:33:47.260661 30033 catalog_manager.cc:5495] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to add3a565be794b079118ae4cb85eb2d6 (127.30.23.129). New cstate: current_term: 1 leader_uuid: "add3a565be794b079118ae4cb85eb2d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6467aa115f2046b582c96b62cd598453" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "add3a565be794b079118ae4cb85eb2d6" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 38045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a802b1bf0ea4629a7abbf5880eaf775" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 43157 } health_report { overall_health: UNKNOWN } } }
I20260809 16:33:47.271803 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.268683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54578 (local address 127.30.23.129:38045)
0809 16:33:47.268760 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.268762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.268772 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:33:47.268848 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.268849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.268871 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.268972 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:33:47.269309 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.269847 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.270723 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.270971 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.271079 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.271375 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:33:47.271523 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:33:47.271606 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.271707 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.271736 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 16:33:47.274169 30510 raft_consensus.cc:1261] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453 [term 1 FOLLOWER]: Refusing update from remote peer add3a565be794b079118ae4cb85eb2d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:47.274188 30724 raft_consensus.cc:1261] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775 [term 1 FOLLOWER]: Refusing update from remote peer add3a565be794b079118ae4cb85eb2d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:47.274657 30859 consensus_queue.cc:1035] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a802b1bf0ea4629a7abbf5880eaf775" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 43157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:47.274893 30853 consensus_queue.cc:1035] T dbfc7ef560e04adcb3d404fe387ce5c7 P add3a565be794b079118ae4cb85eb2d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6467aa115f2046b582c96b62cd598453" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 40485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:47.279110 30867 mvcc.cc:205] Tried to move back new op lower bound from 7316657058911293440 to 7316657058854998016. Current Snapshot: MvccSnapshot[applied={T|T < 7316657058911293440}]
I20260809 16:33:47.279403 30866 mvcc.cc:205] Tried to move back new op lower bound from 7316657058911293440 to 7316657058854998016. Current Snapshot: MvccSnapshot[applied={T|T < 7316657058911293440}]
I20260809 16:33:47.281038 30033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:47.281165 30033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:47.282066 30033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c68f5bcfcc8b4a518c24f812af4b8bed: Sending DeleteTablet for 3 replicas of tablet dbfc7ef560e04adcb3d404fe387ce5c7
I20260809 16:33:47.282423 30474 tablet_service.cc:1514] Processing DeleteTablet for tablet dbfc7ef560e04adcb3d404fe387ce5c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:47 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:42964
I20260809 16:33:47.282500 30236 tablet_service.cc:1514] Processing DeleteTablet for tablet dbfc7ef560e04adcb3d404fe387ce5c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:47 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:54576
I20260809 16:33:47.282707 30878 tablet_replica.cc:330] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453: stopping tablet replica
I20260809 16:33:47.282852 30878 raft_consensus.cc:2229] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:47.282873 30695 tablet_service.cc:1514] Processing DeleteTablet for tablet dbfc7ef560e04adcb3d404fe387ce5c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:47 UTC) from {username='kudu', principal='kudu/127.30.23.190@KRBTEST.COM'} at 127.0.0.1:40036
I20260809 16:33:47.282944 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 30132
I20260809 16:33:47.283006 30878 raft_consensus.cc:2258] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:47.283622 30878 ts_tablet_manager.cc:1905] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:47.285964 30878 ts_tablet_manager.cc:1918] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:47.286051 30878 log.cc:1196] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/dbfc7ef560e04adcb3d404fe387ce5c7
I20260809 16:33:47.286377 30878 ts_tablet_manager.cc:1939] T dbfc7ef560e04adcb3d404fe387ce5c7 P 6467aa115f2046b582c96b62cd598453: Deleting consensus metadata
I20260809 16:33:47.288504 30010 catalog_manager.cc:4841] TS 6467aa115f2046b582c96b62cd598453 (127.30.23.130:40485): tablet dbfc7ef560e04adcb3d404fe387ce5c7 (table test-table [id=b1be46ebfe1645009fca22b12c067d62]) successfully deleted
I20260809 16:33:47.292140 30880 tablet_replica.cc:330] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775: stopping tablet replica
I20260809 16:33:47.292430 30880 raft_consensus.cc:2229] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:47.292680 30880 raft_consensus.cc:2258] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:47.294570 30880 ts_tablet_manager.cc:1905] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:47.296350 30880 ts_tablet_manager.cc:1918] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:47.296425 30880 log.cc:1196] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/dbfc7ef560e04adcb3d404fe387ce5c7
I20260809 16:33:47.296708 30880 ts_tablet_manager.cc:1939] T dbfc7ef560e04adcb3d404fe387ce5c7 P 3a802b1bf0ea4629a7abbf5880eaf775: Deleting consensus metadata
I20260809 16:33:47.297215 30010 catalog_manager.cc:4841] TS 3a802b1bf0ea4629a7abbf5880eaf775 (127.30.23.131:43157): tablet dbfc7ef560e04adcb3d404fe387ce5c7 (table test-table [id=b1be46ebfe1645009fca22b12c067d62]) successfully deleted
W20260809 16:33:47.300644 30010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:38045 (error 108)
I20260809 16:33:47.300851 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 30372
I20260809 16:33:47.312387 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.312136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:38045 (local address 127.0.0.1:54584)
0809 16:33:47.312235 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:47.312321 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 16:33:47.312651 30010 catalog_manager.cc:4584] TS add3a565be794b079118ae4cb85eb2d6 (127.30.23.129:38045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dbfc7ef560e04adcb3d404fe387ce5c7: Network error: Client connection negotiation failed: client connection to 127.30.23.129:38045: connect: Connection refused (error 111)
I20260809 16:33:47.316987 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 30586
I20260809 16:33:47.328042 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 29988
2026-08-09T16:33:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2527 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T16:33:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:33:47Z Disabled control of system clock
I20260809 16:33:47.367798 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:45717
--webserver_interface=127.30.23.190
--webserver_port=0
--builtin_ntp_servers=127.30.23.148:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:45717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:33:47.452081 30891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:47.452313 30891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:47.452334 30891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:47.453440 30891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:47.453475 30891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:47.453488 30891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:47.453501 30891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:47.453513 30891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:47.454818 30891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:45717
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:45717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:47.455029 30891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:47.455268 30891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:47.460311 30903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:47.460352 30904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:47.460510 30906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:47.460582 30891 server_base.cc:1034] running on GCE node
I20260809 16:33:47.460902 30891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:47.461274 30891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:47.462491 30891 hybrid_clock.cc:648] HybridClock initialized: now 1786293227462467 us; error 38 us; skew 500 ppm
I20260809 16:33:47.463784 30891 webserver.cc:460] Webserver started at http://127.30.23.190:45785/ using document root <none> and password file <none>
I20260809 16:33:47.464002 30891 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:47.464044 30891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:47.464135 30891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:47.465178 30891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "b81ee1dbc80f49ddb2e88a12f0476b84"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "3a68d2a703c04746d557f301a06c8739"
server_key_iv: "1b89e2ffb77cbac6c0e4ca4ca81a4ddd"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.465601 30891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "b81ee1dbc80f49ddb2e88a12f0476b84"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "3a68d2a703c04746d557f301a06c8739"
server_key_iv: "1b89e2ffb77cbac6c0e4ca4ca81a4ddd"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.467049 30891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:47.468017 30916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:47.468361 30891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:47.468487 30891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "b81ee1dbc80f49ddb2e88a12f0476b84"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "3a68d2a703c04746d557f301a06c8739"
server_key_iv: "1b89e2ffb77cbac6c0e4ca4ca81a4ddd"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.468570 30891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:47.478094 30891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:47.478443 30891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:47.478621 30891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:47.483455 30891 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:45717
I20260809 16:33:47.483513 30997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:45717 every 8 connection(s)
I20260809 16:33:47.483893 30891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:47.484900 30998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:47.487339 30998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84: Bootstrap starting.
I20260809 16:33:47.487892 30998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:47.488154 30998 log.cc:824] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:47.488809 30998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84: No bootstrap required, opened a new log
I20260809 16:33:47.490494 30998 raft_consensus.cc:348] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b81ee1dbc80f49ddb2e88a12f0476b84" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45717 } }
I20260809 16:33:47.490639 30998 raft_consensus.cc:374] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:47.490662 30998 raft_consensus.cc:729] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b81ee1dbc80f49ddb2e88a12f0476b84, State: Initialized, Role: FOLLOWER
I20260809 16:33:47.490763 30998 consensus_queue.cc:260] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b81ee1dbc80f49ddb2e88a12f0476b84" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45717 } }
I20260809 16:33:47.490818 30998 raft_consensus.cc:388] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:47.490839 30998 raft_consensus.cc:482] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:47.490917 30998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:47.491446 30998 raft_consensus.cc:504] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b81ee1dbc80f49ddb2e88a12f0476b84" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45717 } }
I20260809 16:33:47.491560 30998 leader_election.cc:302] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b81ee1dbc80f49ddb2e88a12f0476b84; no voters:
I20260809 16:33:47.491753 30998 leader_election.cc:288] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:33:47.491803 31006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:47.492033 31006 raft_consensus.cc:686] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [term 1 LEADER]: Becoming Leader. State: Replica: b81ee1dbc80f49ddb2e88a12f0476b84, State: Running, Role: LEADER
I20260809 16:33:47.492048 30998 sys_catalog.cc:564] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:47.492151 31006 consensus_queue.cc:237] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b81ee1dbc80f49ddb2e88a12f0476b84" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45717 } }
I20260809 16:33:47.492601 31010 sys_catalog.cc:455] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b81ee1dbc80f49ddb2e88a12f0476b84. Latest consensus state: current_term: 1 leader_uuid: "b81ee1dbc80f49ddb2e88a12f0476b84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b81ee1dbc80f49ddb2e88a12f0476b84" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45717 } } }
I20260809 16:33:47.492691 31010 sys_catalog.cc:458] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:47.493106 31008 sys_catalog.cc:455] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b81ee1dbc80f49ddb2e88a12f0476b84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b81ee1dbc80f49ddb2e88a12f0476b84" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 45717 } } }
I20260809 16:33:47.493186 31008 sys_catalog.cc:458] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:47.493285 31017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:47.493539 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 30891
I20260809 16:33:47.493675 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:33:47.493858 31017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:47.493912 30814 external_mini_cluster.cc:1428] Setting key 1042f88d29ea6d6cff7dd92b8a46ad13
I20260809 16:33:47.495528 31017 catalog_manager.cc:1347] Generated new cluster ID: e7441aafe67549059a05fffaadb22d86
I20260809 16:33:47.495584 31017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:33:47.499330 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.494617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58056 (local address 127.30.23.190:45717)
0809 16:33:47.494845 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.494854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.494884 (+ 30us) server_negotiation.cc:407] Connection header received
0809 16:33:47.495011 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.495019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.495093 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.495232 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:47.495532 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.496460 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.497534 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.497921 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.498010 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.498653 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:47.498656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:47.498656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:47.498824 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:47.498854 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:47.498865 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:47.498916 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.498998 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.499084 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
I20260809 16:33:47.513161 31017 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:33:47.513756 31017 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:47.520483 31017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84: Generated new TSK 0
I20260809 16:33:47.520686 31017 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:33:47.524674 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.129:0
--local_ip_for_outbound_sockets=127.30.23.129
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45717
--builtin_ntp_servers=127.30.23.148:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:33:47.618098 31042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:47.618330 31042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:47.618358 31042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:47.619691 31042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:47.619738 31042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:47.619766 31042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.129
I20260809 16:33:47.621289 31042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.23.129
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.129
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:47.621644 31042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:47.622008 31042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:47.627935 31048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:47.628021 31042 server_base.cc:1034] running on GCE node
W20260809 16:33:47.627931 31049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:47.628029 31053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:47.628594 31042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:47.628903 31042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:47.630093 31042 hybrid_clock.cc:648] HybridClock initialized: now 1786293227630070 us; error 40 us; skew 500 ppm
I20260809 16:33:47.631551 31042 webserver.cc:460] Webserver started at http://127.30.23.129:46039/ using document root <none> and password file <none>
I20260809 16:33:47.631863 31042 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:47.631944 31042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:47.632100 31042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:47.633252 31042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/data/instance:
uuid: "57f00a21ac9841be91070d70367086c3"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "5a70c90cb986ec838e77ccd355a97899"
server_key_iv: "f01c7432824949b7a7f4674650a6aa6c"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.633718 31042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance:
uuid: "57f00a21ac9841be91070d70367086c3"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "5a70c90cb986ec838e77ccd355a97899"
server_key_iv: "f01c7432824949b7a7f4674650a6aa6c"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.635203 31042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:47.636119 31059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:47.636370 31042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:47.636507 31042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/wal
uuid: "57f00a21ac9841be91070d70367086c3"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "5a70c90cb986ec838e77ccd355a97899"
server_key_iv: "f01c7432824949b7a7f4674650a6aa6c"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.636598 31042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:47.647809 31042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:47.648195 31042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:47.648375 31042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:47.648649 31042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:47.649037 31042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:47.649097 31042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:47.649144 31042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:47.649174 31042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:47.655645 31042 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.129:37685
I20260809 16:33:47.655720 31250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.129:37685 every 8 connection(s)
I20260809 16:33:47.656100 31042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/data/info.pb
I20260809 16:33:47.660996 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 31042
I20260809 16:33:47.661130 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-0/wal/instance
I20260809 16:33:47.661478 30814 external_mini_cluster.cc:1428] Setting key 705ae32693acc6a9a45de6f97f8352b3
I20260809 16:33:47.662531 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.657034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.129:45667 (local address 127.30.23.190:45717)
0809 16:33:47.657120 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.657123 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.657394 (+ 271us) server_negotiation.cc:407] Connection header received
0809 16:33:47.658038 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.658040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.658058 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.658116 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:47.658910 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.659482 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.660732 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.661031 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.661114 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.662233 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:47.662234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:47.662235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:47.662240 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:47.662255 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:47.662259 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:47.662314 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.662439 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.662470 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 16:33:47.662676 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.130:0
--local_ip_for_outbound_sockets=127.30.23.130
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45717
--builtin_ntp_servers=127.30.23.148:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:33:47.662809 31254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.657014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:45717 (local address 127.30.23.129:45667)
0809 16:33:47.657218 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:47.657256 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:47.657909 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:47.658197 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:47.658201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:47.658210 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:47.658792 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.658798 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.659606 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.659607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.660610 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.660613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.661159 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.661160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.661326 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.662072 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:33:47.662073 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:47.662112 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:33:47.662144 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:47.662317 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:47.662323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:47.662327 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:47.662425 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 16:33:47.662548 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260809 16:33:47.663482 31252 heartbeater.cc:347] Connected to a master server at 127.30.23.190:45717
I20260809 16:33:47.663592 31252 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:47.663805 31252 heartbeater.cc:511] Master 127.30.23.190:45717 requested a full tablet report, sending...
I20260809 16:33:47.664321 30943 ts_manager.cc:194] Registered new tserver with Master: 57f00a21ac9841be91070d70367086c3 (127.30.23.129:37685)
I20260809 16:33:47.665009 30943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.23.129:45667
W20260809 16:33:47.753540 31256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:47.753756 31256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:47.753778 31256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:47.755080 31256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:47.755146 31256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:47.755254 31256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.130
I20260809 16:33:47.756832 31256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.23.130
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.130
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:47.757068 31256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:47.757377 31256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:47.763347 31268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:47.763381 31264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:47.763381 31265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:47.763506 31256 server_base.cc:1034] running on GCE node
I20260809 16:33:47.763885 31256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:47.764155 31256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:47.765432 31256 hybrid_clock.cc:648] HybridClock initialized: now 1786293227765405 us; error 36 us; skew 500 ppm
I20260809 16:33:47.766785 31256 webserver.cc:460] Webserver started at http://127.30.23.130:35705/ using document root <none> and password file <none>
I20260809 16:33:47.767086 31256 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:47.767158 31256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:47.767293 31256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:47.768405 31256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/data/instance:
uuid: "e743197327b44a18ae68c69299d14e5c"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "175d0d6ec36845a501f5a6f9a334de4b"
server_key_iv: "f0469a43a7b4d92b15419a8c11a10452"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.768769 31256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance:
uuid: "e743197327b44a18ae68c69299d14e5c"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "175d0d6ec36845a501f5a6f9a334de4b"
server_key_iv: "f0469a43a7b4d92b15419a8c11a10452"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.770278 31256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:33:47.771158 31277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:47.771473 31256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:47.771603 31256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/wal
uuid: "e743197327b44a18ae68c69299d14e5c"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "175d0d6ec36845a501f5a6f9a334de4b"
server_key_iv: "f0469a43a7b4d92b15419a8c11a10452"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.771700 31256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:47.785658 31256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:47.785979 31256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:47.786087 31256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:47.786286 31256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:47.786631 31256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:47.786666 31256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:47.786725 31256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:47.786784 31256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:47.793097 31256 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.130:43049
I20260809 16:33:47.793160 31475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.130:43049 every 8 connection(s)
I20260809 16:33:47.793557 31256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/data/info.pb
I20260809 16:33:47.799466 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 31256
I20260809 16:33:47.799630 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/wal/instance
I20260809 16:33:47.799852 30814 external_mini_cluster.cc:1428] Setting key 3d772744e9426f8f2bdf8cd3891ef461
I20260809 16:33:47.799961 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.794386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.130:34911 (local address 127.30.23.190:45717)
0809 16:33:47.794480 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.794483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.794793 (+ 310us) server_negotiation.cc:407] Connection header received
0809 16:33:47.795465 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.795466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.795478 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.795545 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:47.796283 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.796794 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.798045 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.798238 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.798306 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.799551 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:47.799553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:47.799554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:47.799561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:47.799586 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:47.799591 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:47.799665 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.799831 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.799868 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 16:33:47.800238 31478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.794402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:45717 (local address 127.30.23.130:34911)
0809 16:33:47.794656 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:47.794693 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:47.795328 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:47.795606 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:47.795610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:47.795618 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:47.796175 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.796180 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.796925 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.796926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.797939 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.797943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.798352 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.798353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.798505 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.799357 (+ 852us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:33:47.799358 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:47.799392 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:33:47.799420 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:47.799702 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:47.799713 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:47.799722 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 16:33:47.799808 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:33:47.799937 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":106,"threads_started":1}
I20260809 16:33:47.800781 31476 heartbeater.cc:347] Connected to a master server at 127.30.23.190:45717
I20260809 16:33:47.800879 31476 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:47.801044 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.23.131:0
--local_ip_for_outbound_sockets=127.30.23.131
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45717
--builtin_ntp_servers=127.30.23.148:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:33:47.801108 31476 heartbeater.cc:511] Master 127.30.23.190:45717 requested a full tablet report, sending...
I20260809 16:33:47.801590 30943 ts_manager.cc:194] Registered new tserver with Master: e743197327b44a18ae68c69299d14e5c (127.30.23.130:43049)
I20260809 16:33:47.802107 30943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.23.130:34911
W20260809 16:33:47.892457 31479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:47.892798 31479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:47.892824 31479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:47.894109 31479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:47.894155 31479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:47.894182 31479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.23.131
I20260809 16:33:47.895574 31479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.23.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.23.131
--webserver_port=0
--tserver_master_addrs=127.30.23.190:45717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.23.131
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:47.895782 31479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:47.896071 31479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:47.902076 31488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:47.902087 31494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:47.902092 31490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:47.902465 31479 server_base.cc:1034] running on GCE node
I20260809 16:33:47.902750 31479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:47.903046 31479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:47.904273 31479 hybrid_clock.cc:648] HybridClock initialized: now 1786293227904217 us; error 72 us; skew 500 ppm
I20260809 16:33:47.905978 31479 webserver.cc:460] Webserver started at http://127.30.23.131:36601/ using document root <none> and password file <none>
I20260809 16:33:47.906270 31479 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:47.906351 31479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:47.906525 31479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:47.907929 31479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/data/instance:
uuid: "39663a4b728843ec85c2e7400456c549"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "e5718364b46070c43ad6613b66d4b2a1"
server_key_iv: "35385a0b099393aa7ab026703fa93790"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.908409 31479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance:
uuid: "39663a4b728843ec85c2e7400456c549"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "e5718364b46070c43ad6613b66d4b2a1"
server_key_iv: "35385a0b099393aa7ab026703fa93790"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.910154 31479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260809 16:33:47.911095 31500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:47.911387 31479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:47.911546 31479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/wal
uuid: "39663a4b728843ec85c2e7400456c549"
format_stamp: "Formatted at 2026-08-09 16:33:47 on dist-test-slave-j2vl"
server_key: "e5718364b46070c43ad6613b66d4b2a1"
server_key_iv: "35385a0b099393aa7ab026703fa93790"
server_key_version: "encryptionkey@0"
I20260809 16:33:47.911639 31479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:47.924926 31479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:47.925298 31479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:47.925506 31479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:47.925787 31479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:33:47.926162 31479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:33:47.926222 31479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:47.926272 31479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:33:47.926309 31479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:47.932992 31479 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.131:42165
I20260809 16:33:47.933034 31681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.131:42165 every 8 connection(s)
I20260809 16:33:47.933496 31479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/data/info.pb
I20260809 16:33:47.937590 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 31479
I20260809 16:33:47.937695 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/wal/instance
I20260809 16:33:47.937937 30814 external_mini_cluster.cc:1428] Setting key cf5ba94e9e4a5aee10fc4b114cfe988b
I20260809 16:33:47.940164 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.934334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:53653 (local address 127.30.23.190:45717)
0809 16:33:47.934438 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.934441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.934670 (+ 229us) server_negotiation.cc:407] Connection header received
0809 16:33:47.935283 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.935284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.935300 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.935353 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:47.936123 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.936695 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.938053 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.938407 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.938491 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.939848 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:47.939850 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:47.939851 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:47.939855 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:47.939879 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:47.939885 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:47.939948 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.940073 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.940102 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 16:33:47.940455 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.934297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:45717 (local address 127.30.23.131:53653)
0809 16:33:47.934535 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:47.934573 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:47.935150 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:47.935430 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:47.935434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:47.935445 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:33:47.936016 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.936021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.936822 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.936823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.937932 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.937935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.938747 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.938748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.938951 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.939617 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:33:47.939618 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:33:47.939663 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:33:47.939705 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:33:47.939976 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:33:47.939990 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:33:47.939993 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:33:47.940058 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 16:33:47.940175 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":102,"threads_started":1}
I20260809 16:33:47.940965 31682 heartbeater.cc:347] Connected to a master server at 127.30.23.190:45717
I20260809 16:33:47.941064 31682 heartbeater.cc:464] Registering TS with master...
I20260809 16:33:47.941219 31682 heartbeater.cc:511] Master 127.30.23.190:45717 requested a full tablet report, sending...
I20260809 16:33:47.941675 30943 ts_manager.cc:194] Registered new tserver with Master: 39663a4b728843ec85c2e7400456c549 (127.30.23.131:42165)
I20260809 16:33:47.942095 30943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.23.131:53653
I20260809 16:33:47.950841 30814 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:33:47.956152 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.952485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.30.23.190:45717)
0809 16:33:47.952633 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.952636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.952770 (+ 134us) server_negotiation.cc:407] Connection header received
0809 16:33:47.952858 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.952859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.952876 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.952964 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:47.953187 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.953822 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.954669 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.954905 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.954995 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.955824 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:47.955826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:47.955826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:47.955833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:47.955857 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:47.955862 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:47.955931 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.956007 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.956040 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260809 16:33:47.958343 30943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:33:47.960609 30943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:33:47.968564 31699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.963794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.30.23.130:43049)
0809 16:33:47.964252 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.964257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.964561 (+ 304us) server_negotiation.cc:407] Connection header received
0809 16:33:47.964618 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.964621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.964638 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.964709 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:47.964965 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.966124 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.967288 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.968090 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.968172 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.968289 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.968413 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.968455 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":130,"threads_started":1}
I20260809 16:33:47.968569 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.963540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:43049 (local address 127.0.0.1:39978)
0809 16:33:47.964329 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:47.964342 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:47.964413 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:47.964805 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:47.964805 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:47.964807 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:47.964875 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.964877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.966228 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.966229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.967200 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.967203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.968184 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.968185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.968281 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.968298 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 16:33:47.968396 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 16:33:47.968468 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":25,"thread_start_us":104,"threads_started":1}
I20260809 16:33:47.968751 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.964044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43442 (local address 127.30.23.131:42165)
0809 16:33:47.964265 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.964270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.964480 (+ 210us) server_negotiation.cc:407] Connection header received
0809 16:33:47.964534 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.964536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.964549 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.964647 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:47.964865 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.965839 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.967490 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.968213 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.968276 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.968348 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.968615 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.968664 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":97,"threads_started":1}
I20260809 16:33:47.969069 31700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.963859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34640 (local address 127.30.23.129:37685)
0809 16:33:47.964266 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:47.964270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:47.964300 (+ 30us) server_negotiation.cc:407] Connection header received
0809 16:33:47.964359 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:47.964361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:47.964541 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:47.964665 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:47.964919 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.966124 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.967618 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.968378 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.968449 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.968542 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:47.968937 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 16:33:47.968983 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":103,"threads_started":1}
I20260809 16:33:47.968865 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.963540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:37685 (local address 127.0.0.1:34640)
0809 16:33:47.964086 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:47.964140 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:47.964222 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:47.964755 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:47.964756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:47.964758 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:47.964813 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.964819 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.966217 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.966218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.967526 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.967528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.968583 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.968584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.968654 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.968658 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:47.968718 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:33:47.968771 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":25,"thread_start_us":103,"threads_started":1}
I20260809 16:33:47.969481 31697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:47.963642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.131:42165 (local address 127.0.0.1:43442)
0809 16:33:47.964357 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:47.964367 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:47.964416 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:47.964658 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:47.964662 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:47.964664 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:47.964749 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.964752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.965939 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.965940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.967391 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:47.967394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:47.968388 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:47.968389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:47.968453 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:47.968457 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:47.968508 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:33:47.968552 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":20,"thread_start_us":144,"threads_started":1}
I20260809 16:33:47.970057 31368 tablet_service.cc:1467] Processing CreateTablet for tablet a51381dbd07a428fac897e9798365060 (DEFAULT_TABLE table=test-table [id=b89e964353f14aefbdd812227b41efb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:47.970193 31141 tablet_service.cc:1467] Processing CreateTablet for tablet a51381dbd07a428fac897e9798365060 (DEFAULT_TABLE table=test-table [id=b89e964353f14aefbdd812227b41efb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:47.970497 31368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a51381dbd07a428fac897e9798365060. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:47.970497 31141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a51381dbd07a428fac897e9798365060. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:47.974990 31704 tablet_bootstrap.cc:492] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c: Bootstrap starting.
I20260809 16:33:47.974900 31577 tablet_service.cc:1467] Processing CreateTablet for tablet a51381dbd07a428fac897e9798365060 (DEFAULT_TABLE table=test-table [id=b89e964353f14aefbdd812227b41efb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:33:47.975342 31577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a51381dbd07a428fac897e9798365060. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:47.975734 31704 tablet_bootstrap.cc:654] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:47.976063 31704 log.cc:824] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:47.976428 31702 tablet_bootstrap.cc:492] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3: Bootstrap starting.
I20260809 16:33:47.976949 31704 tablet_bootstrap.cc:492] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c: No bootstrap required, opened a new log
I20260809 16:33:47.977034 31704 ts_tablet_manager.cc:1397] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:33:47.977056 31702 tablet_bootstrap.cc:654] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:47.978281 31702 log.cc:824] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:47.979152 31702 tablet_bootstrap.cc:492] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3: No bootstrap required, opened a new log
I20260809 16:33:47.979136 31709 tablet_bootstrap.cc:492] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549: Bootstrap starting.
I20260809 16:33:47.979233 31702 ts_tablet_manager.cc:1397] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:33:47.979768 31704 raft_consensus.cc:348] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57f00a21ac9841be91070d70367086c3" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37685 } } peers { permanent_uuid: "39663a4b728843ec85c2e7400456c549" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 42165 } } peers { permanent_uuid: "e743197327b44a18ae68c69299d14e5c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43049 } }
I20260809 16:33:47.979970 31704 raft_consensus.cc:374] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:47.980024 31704 raft_consensus.cc:729] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e743197327b44a18ae68c69299d14e5c, State: Initialized, Role: FOLLOWER
I20260809 16:33:47.980152 31709 tablet_bootstrap.cc:654] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:47.980175 31704 consensus_queue.cc:260] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57f00a21ac9841be91070d70367086c3" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37685 } } peers { permanent_uuid: "39663a4b728843ec85c2e7400456c549" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 42165 } } peers { permanent_uuid: "e743197327b44a18ae68c69299d14e5c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43049 } }
I20260809 16:33:47.980486 31476 heartbeater.cc:503] Master 127.30.23.190:45717 was elected leader, sending a full tablet report...
I20260809 16:33:47.980466 31704 ts_tablet_manager.cc:1428] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 16:33:47.980629 31709 log.cc:824] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:47.981489 31709 tablet_bootstrap.cc:492] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549: No bootstrap required, opened a new log
I20260809 16:33:47.981566 31709 ts_tablet_manager.cc:1397] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:33:47.982365 31702 raft_consensus.cc:348] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57f00a21ac9841be91070d70367086c3" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37685 } } peers { permanent_uuid: "39663a4b728843ec85c2e7400456c549" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 42165 } } peers { permanent_uuid: "e743197327b44a18ae68c69299d14e5c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43049 } }
I20260809 16:33:47.982522 31702 raft_consensus.cc:374] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:47.982545 31702 raft_consensus.cc:729] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f00a21ac9841be91070d70367086c3, State: Initialized, Role: FOLLOWER
I20260809 16:33:47.982709 31702 consensus_queue.cc:260] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57f00a21ac9841be91070d70367086c3" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37685 } } peers { permanent_uuid: "39663a4b728843ec85c2e7400456c549" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 42165 } } peers { permanent_uuid: "e743197327b44a18ae68c69299d14e5c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43049 } }
I20260809 16:33:47.983059 31702 ts_tablet_manager.cc:1428] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 16:33:47.983114 31252 heartbeater.cc:503] Master 127.30.23.190:45717 was elected leader, sending a full tablet report...
I20260809 16:33:47.985028 31709 raft_consensus.cc:348] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57f00a21ac9841be91070d70367086c3" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37685 } } peers { permanent_uuid: "39663a4b728843ec85c2e7400456c549" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 42165 } } peers { permanent_uuid: "e743197327b44a18ae68c69299d14e5c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43049 } }
I20260809 16:33:47.985193 31709 raft_consensus.cc:374] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:47.985219 31709 raft_consensus.cc:729] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39663a4b728843ec85c2e7400456c549, State: Initialized, Role: FOLLOWER
I20260809 16:33:47.985353 31709 consensus_queue.cc:260] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57f00a21ac9841be91070d70367086c3" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37685 } } peers { permanent_uuid: "39663a4b728843ec85c2e7400456c549" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 42165 } } peers { permanent_uuid: "e743197327b44a18ae68c69299d14e5c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43049 } }
I20260809 16:33:47.985734 31709 ts_tablet_manager.cc:1428] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260809 16:33:47.985782 31682 heartbeater.cc:503] Master 127.30.23.190:45717 was elected leader, sending a full tablet report...
W20260809 16:33:48.044189 31477 tablet.cc:2369] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:33:48.146286 31714 raft_consensus.cc:482] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:33:48.146489 31714 raft_consensus.cc:504] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57f00a21ac9841be91070d70367086c3" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37685 } } peers { permanent_uuid: "39663a4b728843ec85c2e7400456c549" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 42165 } } peers { permanent_uuid: "e743197327b44a18ae68c69299d14e5c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43049 } }
I20260809 16:33:48.146968 31714 leader_election.cc:288] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57f00a21ac9841be91070d70367086c3 (127.30.23.129:37685), e743197327b44a18ae68c69299d14e5c (127.30.23.130:43049)
I20260809 16:33:48.150893 31700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:48.147253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:34097 (local address 127.30.23.129:37685)
0809 16:33:48.147329 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:48.147332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:48.147345 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:33:48.147448 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:48.147449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:48.147473 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:48.147535 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:48.147764 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:48.148309 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:48.149885 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:48.150487 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:48.150567 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:48.150576 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:48.150777 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 16:33:48.150820 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 16:33:48.150939 31699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:48.147331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.23.131:46577 (local address 127.30.23.130:43049)
0809 16:33:48.147395 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:48.147397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:48.147573 (+ 176us) server_negotiation.cc:407] Connection header received
0809 16:33:48.147607 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:48.147608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:48.147621 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:48.147697 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:33:48.147908 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:48.148463 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:48.149914 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:48.150477 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:48.150567 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:48.150576 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:48.150777 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 16:33:48.150821 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:33:48.150935 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:48.147206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:37685 (local address 127.30.23.131:34097)
0809 16:33:48.147272 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:48.147285 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:48.147338 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:48.147592 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:48.147593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:48.147595 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:48.147674 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:48.147677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:48.148423 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:48.148424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:48.149770 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:48.149773 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:48.150594 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:48.150595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:48.150682 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:48.150686 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:33:48.150757 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 16:33:48.150812 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":8}
I20260809 16:33:48.150941 31715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:48.147220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.130:43049 (local address 127.30.23.131:46577)
0809 16:33:48.147414 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:48.147424 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:33:48.147493 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:33:48.147740 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:33:48.147741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:33:48.147743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:33:48.147825 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:48.147827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:48.148540 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:48.148541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:48.149769 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:33:48.149773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:33:48.150596 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:33:48.150597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:33:48.150685 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:48.150688 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:33:48.150757 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 16:33:48.150808 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":29,"thread_start_us":116,"threads_started":1}
I20260809 16:33:48.151467 31173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a51381dbd07a428fac897e9798365060" candidate_uuid: "39663a4b728843ec85c2e7400456c549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f00a21ac9841be91070d70367086c3" is_pre_election: true
I20260809 16:33:48.151566 31401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a51381dbd07a428fac897e9798365060" candidate_uuid: "39663a4b728843ec85c2e7400456c549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e743197327b44a18ae68c69299d14e5c" is_pre_election: true
I20260809 16:33:48.151684 31401 raft_consensus.cc:2454] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39663a4b728843ec85c2e7400456c549 in term 0.
I20260809 16:33:48.151685 31173 raft_consensus.cc:2454] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39663a4b728843ec85c2e7400456c549 in term 0.
I20260809 16:33:48.151925 31504 leader_election.cc:302] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39663a4b728843ec85c2e7400456c549, e743197327b44a18ae68c69299d14e5c; no voters:
I20260809 16:33:48.152063 31714 raft_consensus.cc:2781] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:33:48.152176 31714 raft_consensus.cc:482] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:33:48.152215 31714 raft_consensus.cc:3037] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:48.153097 31714 raft_consensus.cc:504] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57f00a21ac9841be91070d70367086c3" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37685 } } peers { permanent_uuid: "39663a4b728843ec85c2e7400456c549" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 42165 } } peers { permanent_uuid: "e743197327b44a18ae68c69299d14e5c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43049 } }
I20260809 16:33:48.153265 31714 leader_election.cc:288] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [CANDIDATE]: Term 1 election: Requested vote from peers 57f00a21ac9841be91070d70367086c3 (127.30.23.129:37685), e743197327b44a18ae68c69299d14e5c (127.30.23.130:43049)
I20260809 16:33:48.153462 31173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a51381dbd07a428fac897e9798365060" candidate_uuid: "39663a4b728843ec85c2e7400456c549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f00a21ac9841be91070d70367086c3"
I20260809 16:33:48.153472 31401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a51381dbd07a428fac897e9798365060" candidate_uuid: "39663a4b728843ec85c2e7400456c549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e743197327b44a18ae68c69299d14e5c"
I20260809 16:33:48.153563 31173 raft_consensus.cc:3037] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:48.153563 31401 raft_consensus.cc:3037] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:48.154541 31401 raft_consensus.cc:2454] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39663a4b728843ec85c2e7400456c549 in term 1.
I20260809 16:33:48.154537 31173 raft_consensus.cc:2454] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39663a4b728843ec85c2e7400456c549 in term 1.
I20260809 16:33:48.154765 31506 leader_election.cc:302] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39663a4b728843ec85c2e7400456c549, 57f00a21ac9841be91070d70367086c3; no voters:
I20260809 16:33:48.154892 31714 raft_consensus.cc:2781] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:48.155047 31714 raft_consensus.cc:686] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [term 1 LEADER]: Becoming Leader. State: Replica: 39663a4b728843ec85c2e7400456c549, State: Running, Role: LEADER
I20260809 16:33:48.155146 31714 consensus_queue.cc:237] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57f00a21ac9841be91070d70367086c3" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37685 } } peers { permanent_uuid: "39663a4b728843ec85c2e7400456c549" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 42165 } } peers { permanent_uuid: "e743197327b44a18ae68c69299d14e5c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43049 } }
I20260809 16:33:48.156104 30943 catalog_manager.cc:5495] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39663a4b728843ec85c2e7400456c549 (127.30.23.131). New cstate: current_term: 1 leader_uuid: "39663a4b728843ec85c2e7400456c549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57f00a21ac9841be91070d70367086c3" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39663a4b728843ec85c2e7400456c549" member_type: VOTER last_known_addr { host: "127.30.23.131" port: 42165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e743197327b44a18ae68c69299d14e5c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43049 } health_report { overall_health: UNKNOWN } } }
W20260809 16:33:48.156922 31253 tablet.cc:2369] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:33:48.184167 31683 tablet.cc:2369] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:33:48.199649 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:48.196456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43456 (local address 127.30.23.131:42165)
0809 16:33:48.196548 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:48.196550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:48.196564 (+ 14us) server_negotiation.cc:407] Connection header received
0809 16:33:48.196607 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:48.196608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:48.196627 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:48.196699 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:33:48.196934 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:48.197624 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:48.198522 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:48.198801 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:48.198874 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:48.199130 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:33:48.199246 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:33:48.199328 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:48.199531 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 16:33:48.199554 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 16:33:48.202025 31173 raft_consensus.cc:1261] T a51381dbd07a428fac897e9798365060 P 57f00a21ac9841be91070d70367086c3 [term 1 FOLLOWER]: Refusing update from remote peer 39663a4b728843ec85c2e7400456c549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:48.202025 31401 raft_consensus.cc:1261] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c [term 1 FOLLOWER]: Refusing update from remote peer 39663a4b728843ec85c2e7400456c549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:33:48.202517 31714 consensus_queue.cc:1035] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57f00a21ac9841be91070d70367086c3" member_type: VOTER last_known_addr { host: "127.30.23.129" port: 37685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:48.202716 31717 consensus_queue.cc:1035] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e743197327b44a18ae68c69299d14e5c" member_type: VOTER last_known_addr { host: "127.30.23.130" port: 43049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:33:48.206032 31726 mvcc.cc:205] Tried to move back new op lower bound from 7316657062711218176 to 7316657062524137472. Current Snapshot: MvccSnapshot[applied={T|T < 7316657062711218176 or (T in {7316657062711218176})}]
I20260809 16:33:48.208768 30943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:48.208941 30943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:33:48.209839 30943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b81ee1dbc80f49ddb2e88a12f0476b84: Sending DeleteTablet for 3 replicas of tablet a51381dbd07a428fac897e9798365060
I20260809 16:33:48.210263 31141 tablet_service.cc:1514] Processing DeleteTablet for tablet a51381dbd07a428fac897e9798365060 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:48 UTC) from {username='slave'} at 127.0.0.1:34640
I20260809 16:33:48.210285 31368 tablet_service.cc:1514] Processing DeleteTablet for tablet a51381dbd07a428fac897e9798365060 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:48 UTC) from {username='slave'} at 127.0.0.1:39978
I20260809 16:33:48.210516 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 31042
I20260809 16:33:48.210583 31577 tablet_service.cc:1514] Processing DeleteTablet for tablet a51381dbd07a428fac897e9798365060 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:33:48 UTC) from {username='slave'} at 127.0.0.1:43442
I20260809 16:33:48.238189 31743 tablet_replica.cc:330] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549: stopping tablet replica
I20260809 16:33:48.238257 31741 tablet_replica.cc:330] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c: stopping tablet replica
I20260809 16:33:48.238436 31741 raft_consensus.cc:2229] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:33:48.238610 31743 raft_consensus.cc:2229] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:33:48.238771 31741 raft_consensus.cc:2258] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:48.239001 31743 raft_consensus.cc:2258] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:33:48.239537 31741 ts_tablet_manager.cc:1905] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:48.239827 31743 ts_tablet_manager.cc:1905] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:33:48.241745 31741 ts_tablet_manager.cc:1918] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:48.241752 31743 ts_tablet_manager.cc:1918] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:33:48.241847 31741 log.cc:1196] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-1/wal/wals/a51381dbd07a428fac897e9798365060
I20260809 16:33:48.241984 31743 log.cc:1196] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549: Deleting WAL directory at /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786292963539133-30814-0/minicluster-data/ts-2/wal/wals/a51381dbd07a428fac897e9798365060
I20260809 16:33:48.242129 31741 ts_tablet_manager.cc:1939] T a51381dbd07a428fac897e9798365060 P e743197327b44a18ae68c69299d14e5c: Deleting consensus metadata
I20260809 16:33:48.242400 31743 ts_tablet_manager.cc:1939] T a51381dbd07a428fac897e9798365060 P 39663a4b728843ec85c2e7400456c549: Deleting consensus metadata
I20260809 16:33:48.242730 30920 catalog_manager.cc:4841] TS e743197327b44a18ae68c69299d14e5c (127.30.23.130:43049): tablet a51381dbd07a428fac897e9798365060 (table test-table [id=b89e964353f14aefbdd812227b41efb7]) successfully deleted
I20260809 16:33:48.242851 30917 catalog_manager.cc:4841] TS 39663a4b728843ec85c2e7400456c549 (127.30.23.131:42165): tablet a51381dbd07a428fac897e9798365060 (table test-table [id=b89e964353f14aefbdd812227b41efb7]) successfully deleted
W20260809 16:33:48.244364 30921 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.23.129:37685 (error 108)
I20260809 16:33:48.244959 31697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:48.244789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.129:37685 (local address 127.0.0.1:34652)
0809 16:33:48.244855 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 16:33:48.244902 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.129:37685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 16:33:48.245054 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 31256
W20260809 16:33:48.253595 30921 catalog_manager.cc:4584] TS 57f00a21ac9841be91070d70367086c3 (127.30.23.129:37685): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a51381dbd07a428fac897e9798365060: Network error: Client connection negotiation failed: client connection to 127.30.23.129:37685: connect: Connection refused (error 111)
I20260809 16:33:48.259763 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 31479
I20260809 16:33:48.277716 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 30891
2026-08-09T16:33:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (948 ms)
[----------] 6 tests from EncryptionPolicyTest (10021 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:33:48 dist-test-slave-j2vl krb5kdc[31751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:33:48 dist-test-slave-j2vl krb5kdc[31751](info): set up 2 sockets
Aug 09 16:33:48 dist-test-slave-j2vl krb5kdc[31751](info): commencing operation
krb5kdc: starting...
W20260809 16:33:49.652448 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.345s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:33:49 dist-test-slave-j2vl krb5kdc[31751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:33:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:33:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:33:49.805668 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34495
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:49.900360 31772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:49.900606 31772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:49.900632 31772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:49.901954 31772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:49.902004 31772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:49.902019 31772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:49.902032 31772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:49.902045 31772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:49.903573 31772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34495
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:49.903793 31772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:49.903991 31772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:33:49.909832 31772 server_base.cc:1034] running on GCE node
W20260809 16:33:49.909893 31781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:49.909895 31785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:49.909902 31782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:49.910681 31772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:49.911037 31772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:49.912240 31772 hybrid_clock.cc:648] HybridClock initialized: now 1786293229912214 us; error 43 us; skew 500 ppm
Aug 09 16:33:49 dist-test-slave-j2vl krb5kdc[31751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293229, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:49.920298 31772 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:33:49.920966 31772 webserver.cc:460] Webserver started at http://127.0.0.1:34785/ using document root <none> and password file <none>
I20260809 16:33:49.921178 31772 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:49.921223 31772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:49.921335 31772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:49.922566 31772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "65daf7017b6b4f5192a84c25c6d5fe55"
format_stamp: "Formatted at 2026-08-09 16:33:49 on dist-test-slave-j2vl"
server_key: "9bd99b3ecd508612bacd21decb31283c"
server_key_iv: "c66bb6873f7e49cc17b18cd56a506914"
server_key_version: "encryptionkey@0"
I20260809 16:33:49.922897 31772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "65daf7017b6b4f5192a84c25c6d5fe55"
format_stamp: "Formatted at 2026-08-09 16:33:49 on dist-test-slave-j2vl"
server_key: "9bd99b3ecd508612bacd21decb31283c"
server_key_iv: "c66bb6873f7e49cc17b18cd56a506914"
server_key_version: "encryptionkey@0"
I20260809 16:33:49.924338 31772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:33:49.925280 31796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:49.925695 31772 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:33:49.925843 31772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "65daf7017b6b4f5192a84c25c6d5fe55"
format_stamp: "Formatted at 2026-08-09 16:33:49 on dist-test-slave-j2vl"
server_key: "9bd99b3ecd508612bacd21decb31283c"
server_key_iv: "c66bb6873f7e49cc17b18cd56a506914"
server_key_version: "encryptionkey@0"
I20260809 16:33:49.925976 31772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:49.945209 31772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:49.947768 31772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:49.947970 31772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:49.952647 31772 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34495
I20260809 16:33:49.952731 31874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34495 every 8 connection(s)
I20260809 16:33:49.953122 31772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:49.954262 31875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:49.956732 31875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55: Bootstrap starting.
I20260809 16:33:49.957401 31875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:49.957765 31875 log.cc:824] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:49.958528 31875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55: No bootstrap required, opened a new log
I20260809 16:33:49.960232 31875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65daf7017b6b4f5192a84c25c6d5fe55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34495 } }
I20260809 16:33:49.960391 31875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:49.960418 31875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65daf7017b6b4f5192a84c25c6d5fe55, State: Initialized, Role: FOLLOWER
I20260809 16:33:49.960579 31875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65daf7017b6b4f5192a84c25c6d5fe55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34495 } }
I20260809 16:33:49.960664 31875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:49.960687 31875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:49.960739 31875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:49.961328 31875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65daf7017b6b4f5192a84c25c6d5fe55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34495 } }
I20260809 16:33:49.961519 31875 leader_election.cc:302] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65daf7017b6b4f5192a84c25c6d5fe55; no voters:
I20260809 16:33:49.961720 31875 leader_election.cc:288] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:33:49.961954 31878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:49.962137 31878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [term 1 LEADER]: Becoming Leader. State: Replica: 65daf7017b6b4f5192a84c25c6d5fe55, State: Running, Role: LEADER
I20260809 16:33:49.962191 31875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:49.962281 31878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65daf7017b6b4f5192a84c25c6d5fe55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34495 } }
I20260809 16:33:49.962811 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 31772
I20260809 16:33:49.962935 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:33:49.963179 30814 external_mini_cluster.cc:1428] Setting key b1f3b114e77aac3890e70bf4e11b0216
I20260809 16:33:49.963287 31880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65daf7017b6b4f5192a84c25c6d5fe55. Latest consensus state: current_term: 1 leader_uuid: "65daf7017b6b4f5192a84c25c6d5fe55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65daf7017b6b4f5192a84c25c6d5fe55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34495 } } }
I20260809 16:33:49.963382 31880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:49.963636 31893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:49.963774 31879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65daf7017b6b4f5192a84c25c6d5fe55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65daf7017b6b4f5192a84c25c6d5fe55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34495 } } }
I20260809 16:33:49.963936 31879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:49.964443 31893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:49.966228 31893 catalog_manager.cc:1347] Generated new cluster ID: f7696786a0394a399298d9dc600e81e2
I20260809 16:33:49.966296 31893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:33:49 dist-test-slave-j2vl krb5kdc[31751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:33:49.976315 31901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:49.963946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39479 (local address 127.0.0.1:34495)
0809 16:33:49.964159 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:49.964169 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:49.964205 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:33:49.964333 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:49.964339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:49.964383 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:49.964538 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:49.965239 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:49.966647 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:49.967726 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:49.968404 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:49.968525 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:49.971896 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:49.971901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:49.971901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:49.971916 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:49.974114 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:49.974691 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:49.974694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:49.974695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:49.974836 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:49.975241 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:49.975242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:49.975243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:49.975565 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:49.975716 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:49.975945 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 16:33:49.976066 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":68,"threads_started":1}
I20260809 16:33:49.977697 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:33:49.988099 31901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:49.979439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58093 (local address 127.0.0.1:34495)
0809 16:33:49.979488 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:49.979491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:49.979618 (+ 127us) server_negotiation.cc:407] Connection header received
0809 16:33:49.979654 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:49.979655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:49.979667 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:49.979743 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:49.980416 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:49.981328 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:49.982441 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:49.982825 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:49.982906 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:49.984474 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:49.984476 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:49.984476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:49.984485 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:49.986466 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:49.987031 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:49.987032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:49.987033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:49.987112 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:49.987504 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:49.987505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:49.987505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:49.987655 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:49.987770 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:49.987923 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 16:33:49.987997 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:33:49.991354 31893 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:33:49.992111 31893 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:50.006163 31893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65daf7017b6b4f5192a84c25c6d5fe55: Generated new TSK 0
I20260809 16:33:50.006425 31893 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:33:50.017572 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 31772
W20260809 16:33:50.042960 31747 connection.cc:535] client connection to 127.0.0.1:34495 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34495): Connection reset by peer (error 104)
2026-08-09T16:33:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1750 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:33:50 dist-test-slave-j2vl krb5kdc[31930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:33:50 dist-test-slave-j2vl krb5kdc[31930](info): set up 2 sockets
Aug 09 16:33:50 dist-test-slave-j2vl krb5kdc[31930](info): commencing operation
krb5kdc: starting...
W20260809 16:33:51.386720 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:33:51 dist-test-slave-j2vl krb5kdc[31930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:33:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:33:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:33:51.537688 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41981
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:51.625924 31953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:51.626205 31953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:51.626248 31953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:51.627730 31953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:51.627853 31953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:51.627884 31953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:51.627916 31953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:51.627943 31953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:51.629937 31953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41981
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:51.630276 31953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:51.630657 31953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:51.636282 31966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:51.636348 31962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:51.636518 31953 server_base.cc:1034] running on GCE node
W20260809 16:33:51.636307 31964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:51.637171 31953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:51.637576 31953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:51.638908 31953 hybrid_clock.cc:648] HybridClock initialized: now 1786293231638867 us; error 86 us; skew 500 ppm
Aug 09 16:33:51 dist-test-slave-j2vl krb5kdc[31930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293231, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:51.641891 31953 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:33:51.642452 31953 webserver.cc:460] Webserver started at http://127.0.0.1:35961/ using document root <none> and password file <none>
I20260809 16:33:51.642737 31953 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:51.642819 31953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:51.642983 31953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:51.644250 31953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "dcb419cda13042e3bcc5a0e706398d1c"
format_stamp: "Formatted at 2026-08-09 16:33:51 on dist-test-slave-j2vl"
server_key: "a8f6e6c213982b10ca7cc37556ca5ab3"
server_key_iv: "4e83b9a17bd5de1f6272c1d9c6f0d6eb"
server_key_version: "encryptionkey@0"
I20260809 16:33:51.644683 31953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "dcb419cda13042e3bcc5a0e706398d1c"
format_stamp: "Formatted at 2026-08-09 16:33:51 on dist-test-slave-j2vl"
server_key: "a8f6e6c213982b10ca7cc37556ca5ab3"
server_key_iv: "4e83b9a17bd5de1f6272c1d9c6f0d6eb"
server_key_version: "encryptionkey@0"
I20260809 16:33:51.646582 31953 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 16:33:51.647657 31978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:51.647863 31953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:51.647982 31953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "dcb419cda13042e3bcc5a0e706398d1c"
format_stamp: "Formatted at 2026-08-09 16:33:51 on dist-test-slave-j2vl"
server_key: "a8f6e6c213982b10ca7cc37556ca5ab3"
server_key_iv: "4e83b9a17bd5de1f6272c1d9c6f0d6eb"
server_key_version: "encryptionkey@0"
I20260809 16:33:51.648074 31953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:51.660751 31953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:51.663359 31953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:51.663553 31953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:51.668524 31953 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41981
I20260809 16:33:51.668571 32055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41981 every 8 connection(s)
I20260809 16:33:51.668998 31953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:51.670189 32056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:51.672740 32056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c: Bootstrap starting.
I20260809 16:33:51.673444 32056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:51.673795 32056 log.cc:824] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:51.674497 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 31953
I20260809 16:33:51.674636 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:33:51.674808 32056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c: No bootstrap required, opened a new log
I20260809 16:33:51.674889 30814 external_mini_cluster.cc:1428] Setting key 82dccce839b2013ae056e95f7ce07099
I20260809 16:33:51.677911 32056 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb419cda13042e3bcc5a0e706398d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41981 } }
I20260809 16:33:51.678130 32056 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:51.678172 32056 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcb419cda13042e3bcc5a0e706398d1c, State: Initialized, Role: FOLLOWER
I20260809 16:33:51.678323 32056 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb419cda13042e3bcc5a0e706398d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41981 } }
I20260809 16:33:51.678418 32056 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:51.678450 32056 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:51.678491 32056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:51.679302 32056 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb419cda13042e3bcc5a0e706398d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41981 } }
I20260809 16:33:51.679405 32056 leader_election.cc:302] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcb419cda13042e3bcc5a0e706398d1c; no voters:
I20260809 16:33:51.679618 32056 leader_election.cc:288] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:33:51.679675 32067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:51.679942 32067 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [term 1 LEADER]: Becoming Leader. State: Replica: dcb419cda13042e3bcc5a0e706398d1c, State: Running, Role: LEADER
I20260809 16:33:51.679988 32056 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:51.680114 32067 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb419cda13042e3bcc5a0e706398d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41981 } }
I20260809 16:33:51.680696 32068 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcb419cda13042e3bcc5a0e706398d1c. Latest consensus state: current_term: 1 leader_uuid: "dcb419cda13042e3bcc5a0e706398d1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb419cda13042e3bcc5a0e706398d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41981 } } }
I20260809 16:33:51.680823 32068 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:51.681360 32066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:51.675562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36165 (local address 127.0.0.1:41981)
0809 16:33:51.675793 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:51.675802 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:51.675829 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:33:51.676397 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:51.676405 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:51.676461 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:51.676592 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:51.676896 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:51.677903 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:51.679003 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:51.679528 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:51.679658 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:51.680575 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:51.680580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:51.680580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:51.680801 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:51.680845 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:51.680863 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:51.680957 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:51.681092 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 16:33:51.681123 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":134,"threads_started":1}
I20260809 16:33:51.681394 32074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:51.681560 32069 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcb419cda13042e3bcc5a0e706398d1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcb419cda13042e3bcc5a0e706398d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41981 } } }
I20260809 16:33:51.681648 32069 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:51.681954 32074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:51.682502 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:33:51.685016 32074 catalog_manager.cc:1347] Generated new cluster ID: 2ecf27db97724dc9befb5c0b3a0d3756
I20260809 16:33:51.685082 32074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:33:51.688431 32066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:51.685375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56517 (local address 127.0.0.1:41981)
0809 16:33:51.685437 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:51.685439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:51.685582 (+ 143us) server_negotiation.cc:407] Connection header received
0809 16:33:51.685664 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:51.685665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:51.685676 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:51.685734 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:51.685888 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:51.686365 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:51.687118 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:51.687351 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:51.687433 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:51.688151 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:51.688153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:51.688153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:51.688159 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:51.688175 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:51.688180 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:51.688233 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:51.688307 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 16:33:51.688318 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:33:51.697480 32074 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:33:51.698207 32074 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:51.706660 32074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcb419cda13042e3bcc5a0e706398d1c: Generated new TSK 0
I20260809 16:33:51.706912 32074 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:33:51.715480 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 31953
W20260809 16:33:51.734169 31924 connection.cc:535] client connection to 127.0.0.1:41981 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41981): Connection reset by peer (error 104)
2026-08-09T16:33:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1690 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:33:51 dist-test-slave-j2vl krb5kdc[32104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:33:51 dist-test-slave-j2vl krb5kdc[32104](info): set up 2 sockets
Aug 09 16:33:51 dist-test-slave-j2vl krb5kdc[32104](info): commencing operation
krb5kdc: starting...
W20260809 16:33:53.053785 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:33:53 dist-test-slave-j2vl krb5kdc[32104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:33:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:33:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:33:53.201756 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34085
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:53.290350 32123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:53.290644 32123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:53.290688 32123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:53.291937 32123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:53.292012 32123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:53.292037 32123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:53.292059 32123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:53.292088 32123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:53.293709 32123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34085
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:53.294034 32123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:53.294289 32123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:53.300154 32130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:53.300381 32123 server_base.cc:1034] running on GCE node
W20260809 16:33:53.300256 32134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:53.300159 32131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:53.300750 32123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:53.301139 32123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:53.302380 32123 hybrid_clock.cc:648] HybridClock initialized: now 1786293233302340 us; error 54 us; skew 500 ppm
Aug 09 16:33:53 dist-test-slave-j2vl krb5kdc[32104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293233, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:53.305507 32123 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:33:53.306386 32123 webserver.cc:460] Webserver started at http://127.0.0.1:37821/ using document root <none> and password file <none>
I20260809 16:33:53.306694 32123 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:53.306751 32123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:53.307082 32123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:53.308347 32123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "8b21b8cc08a7486f9c2a40240847f72e"
format_stamp: "Formatted at 2026-08-09 16:33:53 on dist-test-slave-j2vl"
server_key: "f2bf53a2cf35b18b32fbfba24b25217c"
server_key_iv: "e0c966be18481ffe495b7c59c267ce21"
server_key_version: "encryptionkey@0"
I20260809 16:33:53.308751 32123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "8b21b8cc08a7486f9c2a40240847f72e"
format_stamp: "Formatted at 2026-08-09 16:33:53 on dist-test-slave-j2vl"
server_key: "f2bf53a2cf35b18b32fbfba24b25217c"
server_key_iv: "e0c966be18481ffe495b7c59c267ce21"
server_key_version: "encryptionkey@0"
I20260809 16:33:53.310788 32123 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 16:33:53.311902 32147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:53.312177 32123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:53.312283 32123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "8b21b8cc08a7486f9c2a40240847f72e"
format_stamp: "Formatted at 2026-08-09 16:33:53 on dist-test-slave-j2vl"
server_key: "f2bf53a2cf35b18b32fbfba24b25217c"
server_key_iv: "e0c966be18481ffe495b7c59c267ce21"
server_key_version: "encryptionkey@0"
I20260809 16:33:53.312356 32123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:53.316505 32123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:53.319314 32123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:53.319505 32123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:53.323771 32123 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34085
I20260809 16:33:53.323982 32236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34085 every 8 connection(s)
I20260809 16:33:53.324236 32123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:53.325409 32237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:53.327947 32237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e: Bootstrap starting.
I20260809 16:33:53.328227 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 32123
I20260809 16:33:53.328380 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:33:53.328616 30814 external_mini_cluster.cc:1428] Setting key d8957988e51f9ba118d1d188610f0b56
I20260809 16:33:53.328780 32237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:53.329160 32237 log.cc:824] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:53.330024 32237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e: No bootstrap required, opened a new log
I20260809 16:33:53.331593 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:53.329287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36627 (local address 127.0.0.1:34085)
0809 16:33:53.329595 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:53.329603 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:53.329630 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:33:53.330165 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:53.330175 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:53.330216 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:53.330347 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:53.331092 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:53.331096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:53.331096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:53.331100 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:53.331142 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:53.331153 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:53.331231 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:53.331313 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 16:33:53.331379 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":187,"threads_started":1}
I20260809 16:33:53.332554 32237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b21b8cc08a7486f9c2a40240847f72e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34085 } }
I20260809 16:33:53.332753 32237 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:53.332804 32237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b21b8cc08a7486f9c2a40240847f72e, State: Initialized, Role: FOLLOWER
I20260809 16:33:53.332980 32237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b21b8cc08a7486f9c2a40240847f72e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34085 } }
I20260809 16:33:53.333081 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:33:53.333115 32237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:53.333151 32237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:53.333202 32237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:53.334124 32237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b21b8cc08a7486f9c2a40240847f72e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34085 } }
I20260809 16:33:53.334271 32237 leader_election.cc:302] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b21b8cc08a7486f9c2a40240847f72e; no voters:
I20260809 16:33:53.334462 32237 leader_election.cc:288] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:33:53.334534 32251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:53.334734 32251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [term 1 LEADER]: Becoming Leader. State: Replica: 8b21b8cc08a7486f9c2a40240847f72e, State: Running, Role: LEADER
I20260809 16:33:53.334867 32251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b21b8cc08a7486f9c2a40240847f72e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34085 } }
I20260809 16:33:53.335243 32237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:53.335382 32254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b21b8cc08a7486f9c2a40240847f72e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b21b8cc08a7486f9c2a40240847f72e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34085 } } }
I20260809 16:33:53.335475 32254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:53.336076 32258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:53.336522 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:53.335017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40127 (local address 127.0.0.1:34085)
0809 16:33:53.335055 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:53.335057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:53.335250 (+ 193us) server_negotiation.cc:407] Connection header received
0809 16:33:53.335358 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:53.335359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:53.335365 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:53.335399 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:53.336159 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:53.336160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:53.336161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:53.336166 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:53.336181 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:53.336185 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:53.336231 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:53.336432 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 16:33:53.336447 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:33:53.335275 32251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b21b8cc08a7486f9c2a40240847f72e. Latest consensus state: current_term: 1 leader_uuid: "8b21b8cc08a7486f9c2a40240847f72e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b21b8cc08a7486f9c2a40240847f72e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34085 } } }
I20260809 16:33:53.336802 32251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:53.336727 32258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:53.339025 32258 catalog_manager.cc:1347] Generated new cluster ID: 7d3c2bcf518943c1a3f1cf684eb9e5a9
I20260809 16:33:53.339093 32258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:33:53.352571 32258 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:33:53.352792 32258 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:53.361511 32258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b21b8cc08a7486f9c2a40240847f72e: Generated new TSK 0
I20260809 16:33:53.361799 32258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:33:53.366546 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 32123
2026-08-09T16:33:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1671 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:33:53 dist-test-slave-j2vl krb5kdc[32285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:33:53 dist-test-slave-j2vl krb5kdc[32285](info): set up 2 sockets
Aug 09 16:33:53 dist-test-slave-j2vl krb5kdc[32285](info): commencing operation
krb5kdc: starting...
W20260809 16:33:54.745214 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:33:54 dist-test-slave-j2vl krb5kdc[32285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:33:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:33:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:33:54.889859 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39391
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:54.980988 32308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:54.981228 32308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:54.981257 32308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:54.982564 32308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:54.982620 32308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:54.982635 32308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:54.982647 32308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:54.982659 32308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:54.984302 32308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:54.984592 32308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:54.984865 32308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:54.990731 32315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:54.990863 32308 server_base.cc:1034] running on GCE node
W20260809 16:33:54.991034 32314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:54.990873 32317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:54.991498 32308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:54.991863 32308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:54.993039 32308 hybrid_clock.cc:648] HybridClock initialized: now 1786293234993012 us; error 41 us; skew 500 ppm
Aug 09 16:33:54 dist-test-slave-j2vl krb5kdc[32285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293234, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:55.001269 32308 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:33:55.001827 32308 webserver.cc:460] Webserver started at http://127.0.0.1:35169/ using document root <none> and password file <none>
I20260809 16:33:55.002080 32308 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:55.002153 32308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:55.002346 32308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:55.003518 32308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "61f45b5476c546609619070b469284e4"
format_stamp: "Formatted at 2026-08-09 16:33:54 on dist-test-slave-j2vl"
server_key: "54eb1e7b9092be6cc0da1860d3bc9638"
server_key_iv: "b584c2aae471fd38f640cfcd0faae86a"
server_key_version: "encryptionkey@0"
I20260809 16:33:55.003975 32308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "61f45b5476c546609619070b469284e4"
format_stamp: "Formatted at 2026-08-09 16:33:54 on dist-test-slave-j2vl"
server_key: "54eb1e7b9092be6cc0da1860d3bc9638"
server_key_iv: "b584c2aae471fd38f640cfcd0faae86a"
server_key_version: "encryptionkey@0"
I20260809 16:33:55.005542 32308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:33:55.006407 32331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:55.006592 32308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:55.006709 32308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "61f45b5476c546609619070b469284e4"
format_stamp: "Formatted at 2026-08-09 16:33:54 on dist-test-slave-j2vl"
server_key: "54eb1e7b9092be6cc0da1860d3bc9638"
server_key_iv: "b584c2aae471fd38f640cfcd0faae86a"
server_key_version: "encryptionkey@0"
I20260809 16:33:55.006806 32308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:55.021540 32308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:55.024619 32308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:55.024819 32308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:55.029423 32308 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39391
I20260809 16:33:55.029486 32421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39391 every 8 connection(s)
I20260809 16:33:55.029875 32308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:55.031075 32422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:55.033609 32422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4: Bootstrap starting.
I20260809 16:33:55.034186 32422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:55.034446 32422 log.cc:824] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:55.035240 32422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4: No bootstrap required, opened a new log
I20260809 16:33:55.036791 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 32308
I20260809 16:33:55.036947 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:33:55.037123 30814 external_mini_cluster.cc:1428] Setting key 7ec13451bab89446eaf0324af996bc12
I20260809 16:33:55.037109 32422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f45b5476c546609619070b469284e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39391 } }
I20260809 16:33:55.037294 32422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:55.037380 32422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f45b5476c546609619070b469284e4, State: Initialized, Role: FOLLOWER
I20260809 16:33:55.037540 32422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f45b5476c546609619070b469284e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39391 } }
I20260809 16:33:55.037653 32422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:55.037688 32422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:55.037734 32422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:55.038523 32422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f45b5476c546609619070b469284e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39391 } }
I20260809 16:33:55.038646 32422 leader_election.cc:302] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61f45b5476c546609619070b469284e4; no voters:
I20260809 16:33:55.038834 32422 leader_election.cc:288] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:33:55.038897 32427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:55.039101 32427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [term 1 LEADER]: Becoming Leader. State: Replica: 61f45b5476c546609619070b469284e4, State: Running, Role: LEADER
I20260809 16:33:55.039137 32422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:55.039208 32427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f45b5476c546609619070b469284e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39391 } }
I20260809 16:33:55.039691 32427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61f45b5476c546609619070b469284e4. Latest consensus state: current_term: 1 leader_uuid: "61f45b5476c546609619070b469284e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f45b5476c546609619070b469284e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39391 } } }
I20260809 16:33:55.039863 32430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61f45b5476c546609619070b469284e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f45b5476c546609619070b469284e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39391 } } }
I20260809 16:33:55.039960 32427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:55.040030 32430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:55.040191 32441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:55.040766 32441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:55.042629 32441 catalog_manager.cc:1347] Generated new cluster ID: f20e2cde33d3421b822edcd1364c6587
I20260809 16:33:55.042691 32441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:33:55 dist-test-slave-j2vl krb5kdc[32285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:33:55.051138 32426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:55.037726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43705 (local address 127.0.0.1:39391)
0809 16:33:55.037950 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:55.037960 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:55.037995 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:33:55.038425 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:55.038434 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:55.038501 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:55.038674 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:55.039539 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:55.041082 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:55.042139 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:55.042738 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:55.042862 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:55.045938 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:55.045943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:55.045943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:55.045958 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:55.048665 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:55.049216 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:55.049220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:55.049220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:55.049402 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:55.049638 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:55.049639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:55.049639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:55.050016 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:55.050629 (+ 613us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:55.050708 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 16:33:55.050882 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":108,"threads_started":1}
I20260809 16:33:55.053073 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:33:55.056232 32441 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:33:55.056847 32441 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:55.062681 32426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:55.054761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43179 (local address 127.0.0.1:39391)
0809 16:33:55.054803 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:55.054805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:55.054966 (+ 161us) server_negotiation.cc:407] Connection header received
0809 16:33:55.055018 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:55.055019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:55.055030 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:55.055092 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:55.055867 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:55.056540 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:55.057677 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:55.057987 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:55.058057 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:55.059681 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:55.059683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:55.059683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:33:55.059692 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:55.061392 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:55.061809 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:55.061810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:55.061811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:55.061866 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:33:55.062047 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:33:55.062048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:33:55.062048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:33:55.062169 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:55.062253 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:55.062464 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 16:33:55.062577 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:33:55.064090 32441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61f45b5476c546609619070b469284e4: Generated new TSK 0
I20260809 16:33:55.064433 32441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:33:55.071732 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 32308
2026-08-09T16:33:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1686 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:33:55 dist-test-slave-j2vl krb5kdc[32478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:33:55 dist-test-slave-j2vl krb5kdc[32478](info): set up 2 sockets
Aug 09 16:33:55 dist-test-slave-j2vl krb5kdc[32478](info): commencing operation
krb5kdc: starting...
W20260809 16:33:56.387197 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:33:56 dist-test-slave-j2vl krb5kdc[32478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:33:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:33:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:33:56.534453 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37623
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:56.623939 32495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:56.624189 32495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:56.624215 32495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:56.625525 32495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:56.625581 32495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:56.625597 32495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:56.625610 32495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:56.625623 32495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:56.627274 32495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:56.627562 32495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:56.627821 32495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:56.633198 32501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:56.633198 32502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:56.633198 32507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:56.633467 32495 server_base.cc:1034] running on GCE node
I20260809 16:33:56.633745 32495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:56.634109 32495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:56.635280 32495 hybrid_clock.cc:648] HybridClock initialized: now 1786293236635268 us; error 48 us; skew 500 ppm
Aug 09 16:33:56 dist-test-slave-j2vl krb5kdc[32478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293236, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:56.638043 32495 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:33:56.638499 32495 webserver.cc:460] Webserver started at http://127.0.0.1:40803/ using document root <none> and password file <none>
I20260809 16:33:56.638748 32495 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:56.638798 32495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:56.638947 32495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:56.640082 32495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "7a609c0856e44a3eab1f47deaf77f953"
format_stamp: "Formatted at 2026-08-09 16:33:56 on dist-test-slave-j2vl"
server_key: "d87883567f1eed453cbbb3150fe2c670"
server_key_iv: "58ea647efdabff9a63586d4e0c061560"
server_key_version: "encryptionkey@0"
I20260809 16:33:56.640473 32495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "7a609c0856e44a3eab1f47deaf77f953"
format_stamp: "Formatted at 2026-08-09 16:33:56 on dist-test-slave-j2vl"
server_key: "d87883567f1eed453cbbb3150fe2c670"
server_key_iv: "58ea647efdabff9a63586d4e0c061560"
server_key_version: "encryptionkey@0"
I20260809 16:33:56.642078 32495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:33:56.643010 32521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:56.643235 32495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:33:56.643358 32495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "7a609c0856e44a3eab1f47deaf77f953"
format_stamp: "Formatted at 2026-08-09 16:33:56 on dist-test-slave-j2vl"
server_key: "d87883567f1eed453cbbb3150fe2c670"
server_key_iv: "58ea647efdabff9a63586d4e0c061560"
server_key_version: "encryptionkey@0"
I20260809 16:33:56.643437 32495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:56.670048 32495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:56.672544 32495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:56.672678 32495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:56.677531 32495 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37623
I20260809 16:33:56.677567 32598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37623 every 8 connection(s)
I20260809 16:33:56.677963 32495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:56.679006 32601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:56.681133 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 32495
I20260809 16:33:56.681411 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:33:56.681627 30814 external_mini_cluster.cc:1428] Setting key f252a97c5534c76f1691993f25c8ec5a
I20260809 16:33:56.681792 32601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953: Bootstrap starting.
I20260809 16:33:56.682566 32601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:56.682875 32601 log.cc:824] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:56.683610 32601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953: No bootstrap required, opened a new log
I20260809 16:33:56.685734 32601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a609c0856e44a3eab1f47deaf77f953" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37623 } }
I20260809 16:33:56.685915 32601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:56.686031 32601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a609c0856e44a3eab1f47deaf77f953, State: Initialized, Role: FOLLOWER
I20260809 16:33:56.686230 32601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a609c0856e44a3eab1f47deaf77f953" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37623 } }
I20260809 16:33:56.686328 32601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:56.686365 32601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:56.686411 32601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:56.687052 32601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a609c0856e44a3eab1f47deaf77f953" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37623 } }
I20260809 16:33:56.687183 32601 leader_election.cc:302] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a609c0856e44a3eab1f47deaf77f953; no voters:
I20260809 16:33:56.687427 32601 leader_election.cc:288] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:33:56.687446 32606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:56.687634 32606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [term 1 LEADER]: Becoming Leader. State: Replica: 7a609c0856e44a3eab1f47deaf77f953, State: Running, Role: LEADER
I20260809 16:33:56.687713 32604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:56.682243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59499 (local address 127.0.0.1:37623)
0809 16:33:56.682463 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:56.682473 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:56.682511 (+ 38us) server_negotiation.cc:407] Connection header received
0809 16:33:56.683096 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:56.683104 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:56.683156 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:56.683309 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:56.683541 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:56.684495 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:56.685316 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:56.685772 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:56.685910 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:56.686714 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:56.686718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:56.686718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:56.686928 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:56.686965 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:56.686976 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:56.687052 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:56.687128 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 16:33:56.687340 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
I20260809 16:33:56.687731 32601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:56.687777 32606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a609c0856e44a3eab1f47deaf77f953" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37623 } }
I20260809 16:33:56.688225 32607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a609c0856e44a3eab1f47deaf77f953. Latest consensus state: current_term: 1 leader_uuid: "7a609c0856e44a3eab1f47deaf77f953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a609c0856e44a3eab1f47deaf77f953" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37623 } } }
I20260809 16:33:56.688313 32607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:56.688468 32608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a609c0856e44a3eab1f47deaf77f953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a609c0856e44a3eab1f47deaf77f953" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37623 } } }
I20260809 16:33:56.688536 32608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:56.688731 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:33:56.689079 32621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:56.689693 32621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:56.691583 32621 catalog_manager.cc:1347] Generated new cluster ID: ab68dbdad70d40989d3abb05e4ac4d0f
I20260809 16:33:56.691660 32621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:33:56.694783 32621 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:33:56.696158 32621 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:56.698453 32604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:56.690408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51927 (local address 127.0.0.1:37623)
0809 16:33:56.690449 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:56.690451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:56.694528 (+ 4077us) server_negotiation.cc:407] Connection header received
0809 16:33:56.694635 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:56.694636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:56.694648 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:56.694734 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:56.694966 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:56.695877 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:56.696951 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:56.697254 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:56.697335 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:33:56.698159 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:56.698160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:56.698160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:56.698167 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:56.698184 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:56.698189 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:56.698244 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:56.698329 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 16:33:56.698372 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 16:33:56.699060 32621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a609c0856e44a3eab1f47deaf77f953: Generated new TSK 0
I20260809 16:33:56.699249 32621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:33:56.706938 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 32495
2026-08-09T16:33:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1638 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:33:56 dist-test-slave-j2vl krb5kdc[32647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:33:56 dist-test-slave-j2vl krb5kdc[32647](info): set up 2 sockets
Aug 09 16:33:56 dist-test-slave-j2vl krb5kdc[32647](info): commencing operation
krb5kdc: starting...
W20260809 16:33:58.037700 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:33:58 dist-test-slave-j2vl krb5kdc[32647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:33:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:33:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:33:58.186223 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42313
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:58.271437 32665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:58.271672 32665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:58.271696 32665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:58.272818 32665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:58.272858 32665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:58.272873 32665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:58.272886 32665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:58.272899 32665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:58.274492 32665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:58.274734 32665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:58.274935 32665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:58.280323 32682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:58.280339 32680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:58.280515 32665 server_base.cc:1034] running on GCE node
W20260809 16:33:58.280359 32684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:58.280866 32665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:58.281191 32665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:58.282344 32665 hybrid_clock.cc:648] HybridClock initialized: now 1786293238282346 us; error 66 us; skew 500 ppm
Aug 09 16:33:58 dist-test-slave-j2vl krb5kdc[32647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293238, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:58.284946 32665 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:33:58.285499 32665 webserver.cc:460] Webserver started at http://127.0.0.1:43965/ using document root <none> and password file <none>
I20260809 16:33:58.285749 32665 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:58.285821 32665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:58.285980 32665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:58.287163 32665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "376c8f882f694593afc3ca1135507da1"
format_stamp: "Formatted at 2026-08-09 16:33:58 on dist-test-slave-j2vl"
server_key: "16232599fd30506be3db687ca785f6ad"
server_key_iv: "a2ee5698fb72a9b233ad7d70af66d115"
server_key_version: "encryptionkey@0"
I20260809 16:33:58.287559 32665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "376c8f882f694593afc3ca1135507da1"
format_stamp: "Formatted at 2026-08-09 16:33:58 on dist-test-slave-j2vl"
server_key: "16232599fd30506be3db687ca785f6ad"
server_key_iv: "a2ee5698fb72a9b233ad7d70af66d115"
server_key_version: "encryptionkey@0"
I20260809 16:33:58.289085 32665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:33:58.290136 32695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:58.290414 32665 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:33:58.290544 32665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "376c8f882f694593afc3ca1135507da1"
format_stamp: "Formatted at 2026-08-09 16:33:58 on dist-test-slave-j2vl"
server_key: "16232599fd30506be3db687ca785f6ad"
server_key_iv: "a2ee5698fb72a9b233ad7d70af66d115"
server_key_version: "encryptionkey@0"
I20260809 16:33:58.290632 32665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:58.294435 32665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:58.296857 32665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:58.297051 32665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:58.301564 32665 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42313
I20260809 16:33:58.301595 314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42313 every 8 connection(s)
I20260809 16:33:58.301975 32665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:58.302469 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 32665
I20260809 16:33:58.302610 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:33:58.302829 30814 external_mini_cluster.cc:1428] Setting key 3c090fb3d71a7a41c9f142568dafdc87
I20260809 16:33:58.303186 316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:58.306017 322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:58.303538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47959 (local address 127.0.0.1:42313)
0809 16:33:58.303748 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:58.303756 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:58.303784 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:33:58.304234 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:58.304241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:58.304266 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:58.304399 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:58.305171 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:58.305176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:58.305176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:58.305181 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:58.305232 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:58.305243 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:58.305587 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:58.305745 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 16:33:58.305799 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":110,"threads_started":1}
I20260809 16:33:58.307204 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:33:58.307477 316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1: Bootstrap starting.
I20260809 16:33:58.308255 316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:58.308661 316 log.cc:824] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:58.309736 316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1: No bootstrap required, opened a new log
I20260809 16:33:58.310146 322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:58.308894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47571 (local address 127.0.0.1:42313)
0809 16:33:58.308992 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:58.308995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:58.309035 (+ 40us) server_negotiation.cc:407] Connection header received
0809 16:33:58.309105 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:58.309106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:58.309111 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:58.309174 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:58.309863 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:33:58.309865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:33:58.309865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:33:58.309871 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:33:58.309889 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:33:58.309894 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:33:58.309970 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:33:58.310049 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 16:33:58.310060 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 16:33:58.312588 316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "376c8f882f694593afc3ca1135507da1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42313 } }
I20260809 16:33:58.312831 316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:58.313030 316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 376c8f882f694593afc3ca1135507da1, State: Initialized, Role: FOLLOWER
I20260809 16:33:58.313232 316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "376c8f882f694593afc3ca1135507da1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42313 } }
I20260809 16:33:58.313509 316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:58.313592 316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:58.313761 316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:58.315205 316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "376c8f882f694593afc3ca1135507da1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42313 } }
I20260809 16:33:58.315322 316 leader_election.cc:302] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 376c8f882f694593afc3ca1135507da1; no voters:
I20260809 16:33:58.315608 316 leader_election.cc:288] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:33:58.315753 335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:58.315927 335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [term 1 LEADER]: Becoming Leader. State: Replica: 376c8f882f694593afc3ca1135507da1, State: Running, Role: LEADER
I20260809 16:33:58.315955 316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:58.316128 335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "376c8f882f694593afc3ca1135507da1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42313 } }
I20260809 16:33:58.316527 338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 376c8f882f694593afc3ca1135507da1. Latest consensus state: current_term: 1 leader_uuid: "376c8f882f694593afc3ca1135507da1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "376c8f882f694593afc3ca1135507da1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42313 } } }
I20260809 16:33:58.316548 337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "376c8f882f694593afc3ca1135507da1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "376c8f882f694593afc3ca1135507da1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42313 } } }
I20260809 16:33:58.316687 338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:58.316766 337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:58.317039 349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:58.317631 349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:58.319614 349 catalog_manager.cc:1347] Generated new cluster ID: 8a52490f40fe4c53b62b462bf49aad24
I20260809 16:33:58.319672 349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:33:58.333207 349 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:33:58.333449 349 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:33:58.339514 349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 376c8f882f694593afc3ca1135507da1: Generated new TSK 0
I20260809 16:33:58.339778 349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:33:58.351775 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 32665
2026-08-09T16:33:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1642 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:33:58 dist-test-slave-j2vl krb5kdc[370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:33:58 dist-test-slave-j2vl krb5kdc[370](info): set up 2 sockets
Aug 09 16:33:58 dist-test-slave-j2vl krb5kdc[370](info): commencing operation
krb5kdc: starting...
W20260809 16:33:59.683635 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:33:59 dist-test-slave-j2vl krb5kdc[370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:33:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:33:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:33:59.833772 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34899
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:33:59.932304 386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:33:59.932605 386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:33:59.932651 386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:33:59.934100 386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:33:59.934194 386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:33:59.934221 386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:33:59.934244 386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:33:59.934273 386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:33:59.936192 386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:33:59.936550 386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:33:59.936827 386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:33:59.942922 394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:33:59.942860 392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:59.943351 386 server_base.cc:1034] running on GCE node
W20260809 16:33:59.943512 396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:33:59.943821 386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:33:59.944198 386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:33:59.945448 386 hybrid_clock.cc:648] HybridClock initialized: now 1786293239945433 us; error 67 us; skew 500 ppm
Aug 09 16:33:59 dist-test-slave-j2vl krb5kdc[370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:33:59.948356 386 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:33:59.949237 386 webserver.cc:460] Webserver started at http://127.0.0.1:36019/ using document root <none> and password file <none>
I20260809 16:33:59.949600 386 fs_manager.cc:359] Metadata directory not provided
I20260809 16:33:59.949694 386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:33:59.949913 386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:33:59.951304 386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "1ac5e99e54514a4d8bf7ce444f4adf39"
format_stamp: "Formatted at 2026-08-09 16:33:59 on dist-test-slave-j2vl"
server_key: "6eff40d2dafabb092c53d2ad3371c8c1"
server_key_iv: "12e828bee8f6e5fed98b55fbbbee644e"
server_key_version: "encryptionkey@0"
I20260809 16:33:59.951673 386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "1ac5e99e54514a4d8bf7ce444f4adf39"
format_stamp: "Formatted at 2026-08-09 16:33:59 on dist-test-slave-j2vl"
server_key: "6eff40d2dafabb092c53d2ad3371c8c1"
server_key_iv: "12e828bee8f6e5fed98b55fbbbee644e"
server_key_version: "encryptionkey@0"
I20260809 16:33:59.953135 386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:33:59.954192 405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:33:59.954481 386 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:33:59.954571 386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "1ac5e99e54514a4d8bf7ce444f4adf39"
format_stamp: "Formatted at 2026-08-09 16:33:59 on dist-test-slave-j2vl"
server_key: "6eff40d2dafabb092c53d2ad3371c8c1"
server_key_iv: "12e828bee8f6e5fed98b55fbbbee644e"
server_key_version: "encryptionkey@0"
I20260809 16:33:59.954633 386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:33:59.974468 386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:33:59.976970 386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:33:59.977101 386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:33:59.981652 386 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34899
I20260809 16:33:59.981714 485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34899 every 8 connection(s)
I20260809 16:33:59.982126 386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:33:59.983150 486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:33:59.987092 486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39: Bootstrap starting.
I20260809 16:33:59.987895 486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39: Neither blocks nor log segments found. Creating new log.
I20260809 16:33:59.988243 486 log.cc:824] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39: Log is configured to *not* fsync() on all Append() calls
I20260809 16:33:59.989283 486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39: No bootstrap required, opened a new log
I20260809 16:33:59.991082 486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac5e99e54514a4d8bf7ce444f4adf39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34899 } }
I20260809 16:33:59.991272 486 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:33:59.991326 486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ac5e99e54514a4d8bf7ce444f4adf39, State: Initialized, Role: FOLLOWER
I20260809 16:33:59.991540 486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac5e99e54514a4d8bf7ce444f4adf39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34899 } }
I20260809 16:33:59.991652 486 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:33:59.991699 486 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:33:59.991755 486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:33:59.992089 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 386
I20260809 16:33:59.992238 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:33:59.992492 30814 external_mini_cluster.cc:1428] Setting key 44d56af8f0d091230679f887195be2eb
I20260809 16:33:59.992514 486 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac5e99e54514a4d8bf7ce444f4adf39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34899 } }
I20260809 16:33:59.992652 486 leader_election.cc:302] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ac5e99e54514a4d8bf7ce444f4adf39; no voters:
I20260809 16:33:59.992879 486 leader_election.cc:288] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:33:59.992916 494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:33:59.993043 494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [term 1 LEADER]: Becoming Leader. State: Replica: 1ac5e99e54514a4d8bf7ce444f4adf39, State: Running, Role: LEADER
I20260809 16:33:59.993192 486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:33:59.993192 494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac5e99e54514a4d8bf7ce444f4adf39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34899 } }
I20260809 16:33:59.993762 495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ac5e99e54514a4d8bf7ce444f4adf39. Latest consensus state: current_term: 1 leader_uuid: "1ac5e99e54514a4d8bf7ce444f4adf39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac5e99e54514a4d8bf7ce444f4adf39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34899 } } }
I20260809 16:33:59.993880 495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:59.993944 497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ac5e99e54514a4d8bf7ce444f4adf39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ac5e99e54514a4d8bf7ce444f4adf39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34899 } } }
I20260809 16:33:59.994033 497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39 [sys.catalog]: This master's current role is: LEADER
I20260809 16:33:59.995121 513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:33:59.995723 513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:33:59.997790 513 catalog_manager.cc:1347] Generated new cluster ID: 7ca1273550924a41a047e68f5a396438
I20260809 16:33:59.997854 513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:34:00 dist-test-slave-j2vl krb5kdc[370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:34:00.013737 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:33:59.993288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:50241 (local address 127.0.0.1:34899)
0809 16:33:59.993499 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 16:33:59.993510 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:33:59.993829 (+ 319us) server_negotiation.cc:407] Connection header received
0809 16:33:59.993985 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:33:59.993994 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:33:59.994068 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:33:59.994234 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:33:59.995285 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:59.996486 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:59.997591 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:33:59.998266 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:33:59.998423 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:00.009248 (+ 10825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:00.009254 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:00.009254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:00.009270 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:00.011247 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:00.011737 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:00.011740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:00.011741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:00.011838 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:00.012092 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:00.012095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:00.012095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:00.012619 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:00.012949 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:00.013153 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 16:34:00.013458 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":102,"threads_started":1}
I20260809 16:34:00.014878 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:00.016475 513 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:00.017194 513 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:00.025614 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:00.016673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:47427 (local address 127.0.0.1:34899)
0809 16:34:00.016713 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:00.016715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:00.016723 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:34:00.016745 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:00.016746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:00.016754 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:00.016805 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:00.017446 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:00.017971 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:00.019260 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:00.019745 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:00.019886 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:00.022132 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:00.022134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:00.022135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:00.022145 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:00.024219 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:00.024687 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:00.024688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:00.024688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:00.024753 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:00.024998 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:00.024999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:00.024999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:00.025126 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:00.025249 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:00.025427 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 16:34:00.025495 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:34:00.029842 513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ac5e99e54514a4d8bf7ce444f4adf39: Generated new TSK 0
I20260809 16:34:00.030102 513 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:00.036366 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 386
W20260809 16:34:00.070677 366 connection.cc:535] client connection to 127.0.0.1:34899 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34899): Connection reset by peer (error 104)
2026-08-09T16:34:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1697 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:00 dist-test-slave-j2vl krb5kdc[538](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:00 dist-test-slave-j2vl krb5kdc[538](info): set up 2 sockets
Aug 09 16:34:00 dist-test-slave-j2vl krb5kdc[538](info): commencing operation
krb5kdc: starting...
W20260809 16:34:01.421507 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:34:01 dist-test-slave-j2vl krb5kdc[538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:34:01.566517 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41939
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:01.656584 558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:01.656828 558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:01.656857 558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:01.658171 558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:01.658228 558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:01.658244 558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:01.658258 558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:01.658270 558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:01.659986 558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:01.660283 558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:01.660554 558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:01.666015 565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:01.666006 566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:01.666098 571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:01.666209 558 server_base.cc:1034] running on GCE node
I20260809 16:34:01.666523 558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:01.666833 558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:01.668005 558 hybrid_clock.cc:648] HybridClock initialized: now 1786293241667974 us; error 42 us; skew 500 ppm
Aug 09 16:34:01 dist-test-slave-j2vl krb5kdc[538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:01.670881 558 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:34:01.671336 558 webserver.cc:460] Webserver started at http://127.0.0.1:41039/ using document root <none> and password file <none>
I20260809 16:34:01.671617 558 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:01.671695 558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:01.671864 558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:01.673035 558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "8efc2e9c5963450483c667709fe3ce95"
format_stamp: "Formatted at 2026-08-09 16:34:01 on dist-test-slave-j2vl"
server_key: "6d37f4ae0eeb56b889b44d8811c463e1"
server_key_iv: "0fbf642452a73aea674c17e35eac1e76"
server_key_version: "encryptionkey@0"
I20260809 16:34:01.673485 558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "8efc2e9c5963450483c667709fe3ce95"
format_stamp: "Formatted at 2026-08-09 16:34:01 on dist-test-slave-j2vl"
server_key: "6d37f4ae0eeb56b889b44d8811c463e1"
server_key_iv: "0fbf642452a73aea674c17e35eac1e76"
server_key_version: "encryptionkey@0"
I20260809 16:34:01.675068 558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:01.676213 585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:01.676514 558 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:01.676738 558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "8efc2e9c5963450483c667709fe3ce95"
format_stamp: "Formatted at 2026-08-09 16:34:01 on dist-test-slave-j2vl"
server_key: "6d37f4ae0eeb56b889b44d8811c463e1"
server_key_iv: "0fbf642452a73aea674c17e35eac1e76"
server_key_version: "encryptionkey@0"
I20260809 16:34:01.676832 558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:01.690827 558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:01.693711 558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:01.693931 558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:01.698637 558 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41939
I20260809 16:34:01.698675 680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41939 every 8 connection(s)
I20260809 16:34:01.699092 558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:01.700333 681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:01.703032 681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95: Bootstrap starting.
I20260809 16:34:01.703367 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 558
I20260809 16:34:01.703523 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:34:01.703749 30814 external_mini_cluster.cc:1428] Setting key 471dde8424c17c92a39e67a23bee49cb
I20260809 16:34:01.703879 681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:01.704222 681 log.cc:824] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:01.705093 681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95: No bootstrap required, opened a new log
I20260809 16:34:01.706825 681 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc2e9c5963450483c667709fe3ce95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41939 } }
I20260809 16:34:01.707015 681 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:01.707067 681 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8efc2e9c5963450483c667709fe3ce95, State: Initialized, Role: FOLLOWER
I20260809 16:34:01.707216 681 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc2e9c5963450483c667709fe3ce95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41939 } }
I20260809 16:34:01.707322 681 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:01.707376 681 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:01.707430 681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:01.708088 681 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc2e9c5963450483c667709fe3ce95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41939 } }
I20260809 16:34:01.708226 681 leader_election.cc:302] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8efc2e9c5963450483c667709fe3ce95; no voters:
I20260809 16:34:01.708458 681 leader_election.cc:288] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:01.708510 689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:01.708669 689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [term 1 LEADER]: Becoming Leader. State: Replica: 8efc2e9c5963450483c667709fe3ce95, State: Running, Role: LEADER
I20260809 16:34:01.708770 689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc2e9c5963450483c667709fe3ce95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41939 } }
I20260809 16:34:01.708848 681 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:01.709116 689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8efc2e9c5963450483c667709fe3ce95. Latest consensus state: current_term: 1 leader_uuid: "8efc2e9c5963450483c667709fe3ce95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc2e9c5963450483c667709fe3ce95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41939 } } }
I20260809 16:34:01.709122 691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8efc2e9c5963450483c667709fe3ce95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc2e9c5963450483c667709fe3ce95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41939 } } }
I20260809 16:34:01.709223 689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:01.709231 691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:01.709524 695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:01.710335 687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:01.704401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:42141 (local address 127.0.0.1:41939)
0809 16:34:01.704615 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:01.704623 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:01.704648 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:34:01.705116 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:01.705123 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:01.705222 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:01.705397 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:01.705627 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:01.706542 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:01.707777 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:01.708236 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:01.708375 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:01.709273 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:01.709277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:01.709278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:01.709623 (+ 345us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:01.709671 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:01.709692 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:01.709772 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:01.710011 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 16:34:01.710045 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":109,"threads_started":1}
I20260809 16:34:01.710119 695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:01.711683 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:01.712067 695 catalog_manager.cc:1347] Generated new cluster ID: c2614d32d6f842cb8eead1d432609763
I20260809 16:34:01.712123 695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:01.720928 687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:01.713853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:50171 (local address 127.0.0.1:41939)
0809 16:34:01.713983 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:01.713985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:01.717595 (+ 3610us) server_negotiation.cc:407] Connection header received
0809 16:34:01.717688 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:01.717689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:01.717699 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:01.717761 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:01.717933 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:01.718599 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:01.719400 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:01.719628 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:01.719719 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:01.720616 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:01.720618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:01.720618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:01.720624 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:01.720641 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:01.720645 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:01.720705 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:01.720781 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 16:34:01.720794 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:34:01.726737 695 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:01.727309 695 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:01.741766 695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8efc2e9c5963450483c667709fe3ce95: Generated new TSK 0
I20260809 16:34:01.742069 695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:01.745896 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 558
W20260809 16:34:01.764910 535 connection.cc:535] client connection to 127.0.0.1:41939 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41939): Connection reset by peer (error 104)
2026-08-09T16:34:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1694 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:01 dist-test-slave-j2vl krb5kdc[732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:01 dist-test-slave-j2vl krb5kdc[732](info): set up 2 sockets
Aug 09 16:34:01 dist-test-slave-j2vl krb5kdc[732](info): commencing operation
krb5kdc: starting...
W20260809 16:34:03.081171 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:34:03 dist-test-slave-j2vl krb5kdc[732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:34:03.226464 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32949
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:03.315107 762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:03.315408 762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:03.315452 762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:03.316704 762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:03.316788 762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:03.316818 762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:03.316847 762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:03.316877 762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:03.318535 762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:03.318857 762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:03.319120 762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:03.324851 773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:03.324855 771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:03.324859 769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:03.325361 762 server_base.cc:1034] running on GCE node
I20260809 16:34:03.325662 762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:03.326035 762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:03.327220 762 hybrid_clock.cc:648] HybridClock initialized: now 1786293243327199 us; error 42 us; skew 500 ppm
Aug 09 16:34:03 dist-test-slave-j2vl krb5kdc[732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293243, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:03.333673 762 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:34:03.334270 762 webserver.cc:460] Webserver started at http://127.0.0.1:40385/ using document root <none> and password file <none>
I20260809 16:34:03.334558 762 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:03.334612 762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:03.334805 762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:03.336053 762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "d234ba1f58594b3ab2475ae18cd9b5ff"
format_stamp: "Formatted at 2026-08-09 16:34:03 on dist-test-slave-j2vl"
server_key: "9fdf36a905111140470cadfbd60f274f"
server_key_iv: "af54863d28239a222c7be68b5485e4be"
server_key_version: "encryptionkey@0"
I20260809 16:34:03.336467 762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "d234ba1f58594b3ab2475ae18cd9b5ff"
format_stamp: "Formatted at 2026-08-09 16:34:03 on dist-test-slave-j2vl"
server_key: "9fdf36a905111140470cadfbd60f274f"
server_key_iv: "af54863d28239a222c7be68b5485e4be"
server_key_version: "encryptionkey@0"
I20260809 16:34:03.338225 762 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 16:34:03.339170 790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:03.339570 762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:03.339670 762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "d234ba1f58594b3ab2475ae18cd9b5ff"
format_stamp: "Formatted at 2026-08-09 16:34:03 on dist-test-slave-j2vl"
server_key: "9fdf36a905111140470cadfbd60f274f"
server_key_iv: "af54863d28239a222c7be68b5485e4be"
server_key_version: "encryptionkey@0"
I20260809 16:34:03.339787 762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:03.343416 762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:03.346477 762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:03.346690 762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:03.351727 762 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32949
I20260809 16:34:03.351928 873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32949 every 8 connection(s)
I20260809 16:34:03.352211 762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:03.353278 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 762
I20260809 16:34:03.353314 874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:03.353436 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:34:03.353655 30814 external_mini_cluster.cc:1428] Setting key b5f51c832f3b3b6a6d2687d1fc250d65
I20260809 16:34:03.356472 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:03.354293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:39003 (local address 127.0.0.1:32949)
0809 16:34:03.354509 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:03.354527 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:03.354557 (+ 30us) server_negotiation.cc:407] Connection header received
0809 16:34:03.355058 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:03.355065 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:03.355093 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:03.355293 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:03.356047 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:03.356058 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:03.356058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:03.356062 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:03.356101 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:03.356112 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:03.356186 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:03.356259 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 16:34:03.356304 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260809 16:34:03.356863 874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff: Bootstrap starting.
I20260809 16:34:03.357831 874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:03.358194 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:03.358179 874 log.cc:824] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:03.359162 874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff: No bootstrap required, opened a new log
I20260809 16:34:03.361220 874 raft_consensus.cc:348] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d234ba1f58594b3ab2475ae18cd9b5ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32949 } }
I20260809 16:34:03.361452 874 raft_consensus.cc:374] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:03.361503 874 raft_consensus.cc:729] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d234ba1f58594b3ab2475ae18cd9b5ff, State: Initialized, Role: FOLLOWER
I20260809 16:34:03.361670 874 consensus_queue.cc:260] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d234ba1f58594b3ab2475ae18cd9b5ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32949 } }
I20260809 16:34:03.361806 874 raft_consensus.cc:388] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:03.361843 874 raft_consensus.cc:482] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:03.361922 874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:03.362823 874 raft_consensus.cc:504] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d234ba1f58594b3ab2475ae18cd9b5ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32949 } }
I20260809 16:34:03.363044 874 leader_election.cc:302] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d234ba1f58594b3ab2475ae18cd9b5ff; no voters:
I20260809 16:34:03.363303 874 leader_election.cc:288] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:03.363353 887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:03.363541 887 raft_consensus.cc:686] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [term 1 LEADER]: Becoming Leader. State: Replica: d234ba1f58594b3ab2475ae18cd9b5ff, State: Running, Role: LEADER
I20260809 16:34:03.363626 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:03.361672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:56347 (local address 127.0.0.1:32949)
0809 16:34:03.361734 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:03.361736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:03.362005 (+ 269us) server_negotiation.cc:407] Connection header received
0809 16:34:03.362142 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:03.362144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:03.362150 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:03.362227 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:03.363277 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:03.363279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:03.363279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:03.363285 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:03.363306 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:03.363311 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:03.363369 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:03.363539 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 16:34:03.363554 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:34:03.363710 887 consensus_queue.cc:237] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d234ba1f58594b3ab2475ae18cd9b5ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32949 } }
I20260809 16:34:03.363632 874 sys_catalog.cc:564] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:03.364385 891 sys_catalog.cc:455] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader d234ba1f58594b3ab2475ae18cd9b5ff. Latest consensus state: current_term: 1 leader_uuid: "d234ba1f58594b3ab2475ae18cd9b5ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d234ba1f58594b3ab2475ae18cd9b5ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32949 } } }
I20260809 16:34:03.364524 891 sys_catalog.cc:458] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:03.364398 890 sys_catalog.cc:455] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d234ba1f58594b3ab2475ae18cd9b5ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d234ba1f58594b3ab2475ae18cd9b5ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32949 } } }
I20260809 16:34:03.364668 890 sys_catalog.cc:458] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:03.364792 896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:03.365609 896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:03.367624 896 catalog_manager.cc:1347] Generated new cluster ID: 9aa76019193b49bdbd1138e51801271d
I20260809 16:34:03.367695 896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:03.384086 896 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:03.384352 896 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:03.391546 896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d234ba1f58594b3ab2475ae18cd9b5ff: Generated new TSK 0
I20260809 16:34:03.391837 896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:03.395361 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 762
2026-08-09T16:34:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1648 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:03 dist-test-slave-j2vl krb5kdc[925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:03 dist-test-slave-j2vl krb5kdc[925](info): set up 2 sockets
Aug 09 16:34:03 dist-test-slave-j2vl krb5kdc[925](info): commencing operation
krb5kdc: starting...
W20260809 16:34:04.746584 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:34:04 dist-test-slave-j2vl krb5kdc[925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:34:04.906158 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:04.999604 949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:04.999897 949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:04.999943 949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:05.001298 949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:05.001411 949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:05.001446 949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:05.001480 949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:05.001507 949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:05.003261 949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:05.003599 949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:05.003871 949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:05.009439 959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:05.009445 966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:05.009513 962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:05.009835 949 server_base.cc:1034] running on GCE node
I20260809 16:34:05.010113 949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:05.010484 949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:05.011660 949 hybrid_clock.cc:648] HybridClock initialized: now 1786293245011633 us; error 52 us; skew 500 ppm
Aug 09 16:34:05 dist-test-slave-j2vl krb5kdc[925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293245, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:05.019057 949 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:34:05.019641 949 webserver.cc:460] Webserver started at http://127.0.0.1:40245/ using document root <none> and password file <none>
I20260809 16:34:05.019901 949 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:05.019976 949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:05.020134 949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:05.021266 949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "8bc2544c627d4c8fae841d21c7ea82fb"
format_stamp: "Formatted at 2026-08-09 16:34:05 on dist-test-slave-j2vl"
server_key: "4079b636d869b2946711b3e3834b54a4"
server_key_iv: "f98129a78934e03dbe50b92725032516"
server_key_version: "encryptionkey@0"
I20260809 16:34:05.021721 949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "8bc2544c627d4c8fae841d21c7ea82fb"
format_stamp: "Formatted at 2026-08-09 16:34:05 on dist-test-slave-j2vl"
server_key: "4079b636d869b2946711b3e3834b54a4"
server_key_iv: "f98129a78934e03dbe50b92725032516"
server_key_version: "encryptionkey@0"
I20260809 16:34:05.023152 949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:05.024025 977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:05.024264 949 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:05.024398 949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "8bc2544c627d4c8fae841d21c7ea82fb"
format_stamp: "Formatted at 2026-08-09 16:34:05 on dist-test-slave-j2vl"
server_key: "4079b636d869b2946711b3e3834b54a4"
server_key_iv: "f98129a78934e03dbe50b92725032516"
server_key_version: "encryptionkey@0"
I20260809 16:34:05.024488 949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:05.046566 949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:05.048879 949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:05.049062 949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:05.053646 949 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33607
I20260809 16:34:05.053802 1061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33607 every 8 connection(s)
I20260809 16:34:05.054116 949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:05.055125 1063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:05.057507 1063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb: Bootstrap starting.
I20260809 16:34:05.058085 1063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:05.058328 1063 log.cc:824] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:05.059088 1063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb: No bootstrap required, opened a new log
I20260809 16:34:05.060684 1063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc2544c627d4c8fae841d21c7ea82fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33607 } }
I20260809 16:34:05.060828 1063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:05.060851 1063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bc2544c627d4c8fae841d21c7ea82fb, State: Initialized, Role: FOLLOWER
I20260809 16:34:05.061000 1063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc2544c627d4c8fae841d21c7ea82fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33607 } }
I20260809 16:34:05.061089 1063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:05.061115 1063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:05.061142 1063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:05.061753 1063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc2544c627d4c8fae841d21c7ea82fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33607 } }
I20260809 16:34:05.061843 1063 leader_election.cc:302] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bc2544c627d4c8fae841d21c7ea82fb; no voters:
I20260809 16:34:05.061975 1063 leader_election.cc:288] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:05.062047 1066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:05.062247 1066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [term 1 LEADER]: Becoming Leader. State: Replica: 8bc2544c627d4c8fae841d21c7ea82fb, State: Running, Role: LEADER
I20260809 16:34:05.062264 1063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:05.062398 1066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc2544c627d4c8fae841d21c7ea82fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33607 } }
I20260809 16:34:05.062892 1071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bc2544c627d4c8fae841d21c7ea82fb. Latest consensus state: current_term: 1 leader_uuid: "8bc2544c627d4c8fae841d21c7ea82fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc2544c627d4c8fae841d21c7ea82fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33607 } } }
I20260809 16:34:05.062983 1071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:05.063032 1069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bc2544c627d4c8fae841d21c7ea82fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc2544c627d4c8fae841d21c7ea82fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33607 } } }
I20260809 16:34:05.063115 1069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:05.063503 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 949
I20260809 16:34:05.063671 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:34:05.063657 1087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:05.063949 30814 external_mini_cluster.cc:1428] Setting key 6a539c1cf24398be4d3b99c9a9617e8e
I20260809 16:34:05.064256 1087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:05.066095 1087 catalog_manager.cc:1347] Generated new cluster ID: 4fdc1f0af39d4de4aae220a12888513e
I20260809 16:34:05.066155 1087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:34:05 dist-test-slave-j2vl krb5kdc[925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:34:05.076897 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:05.064511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:39223 (local address 127.0.0.1:33607)
0809 16:34:05.064717 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:05.064728 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:05.064817 (+ 89us) server_negotiation.cc:407] Connection header received
0809 16:34:05.064995 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:05.065003 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:05.065076 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:05.065234 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:05.066138 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:05.067503 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:05.069023 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:05.069692 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:05.069796 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:05.072921 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:05.072926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:05.072926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:05.072941 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:05.075091 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:05.075557 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:05.075561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:05.075561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:05.075691 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:05.075936 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:05.075937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:05.075937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:05.076266 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:05.076436 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:05.076545 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 16:34:05.076657 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260809 16:34:05.078270 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:05.081332 1087 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:05.081844 1087 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:05.085978 1087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bc2544c627d4c8fae841d21c7ea82fb: Generated new TSK 0
I20260809 16:34:05.086159 1087 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:05.092940 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:05.079737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:40649 (local address 127.0.0.1:33607)
0809 16:34:05.079773 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:05.079775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:05.080987 (+ 1212us) server_negotiation.cc:407] Connection header received
0809 16:34:05.081091 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:05.081092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:05.081103 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:05.081154 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:05.086032 (+ 4878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:05.087205 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:05.087989 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:05.088318 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:05.088391 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:05.089859 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:05.089861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:05.089861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:05.089870 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:05.091476 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:05.091908 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:05.091910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:05.091910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:05.091973 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:05.092262 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:05.092263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:05.092264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:05.092426 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:05.092533 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:05.092793 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 16:34:05.092868 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:34:05.095686 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 949
W20260809 16:34:05.120420 921 connection.cc:535] client connection to 127.0.0.1:33607 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33607): Connection reset by peer (error 104)
2026-08-09T16:34:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1706 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:05 dist-test-slave-j2vl krb5kdc[1113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:05 dist-test-slave-j2vl krb5kdc[1113](info): set up 2 sockets
Aug 09 16:34:05 dist-test-slave-j2vl krb5kdc[1113](info): commencing operation
krb5kdc: starting...
W20260809 16:34:06.410871 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:34:06 dist-test-slave-j2vl krb5kdc[1113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:34:06.556910 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42905
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:06.643746 1145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:06.644001 1145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:06.644040 1145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:06.645262 1145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:06.645337 1145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:06.645397 1145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:06.645449 1145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:06.645478 1145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:06.647047 1145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42905
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:06.647356 1145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:06.647598 1145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:06.652920 1153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:06.652892 1159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:06.653047 1145 server_base.cc:1034] running on GCE node
W20260809 16:34:06.652878 1163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:06.653465 1145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:06.653777 1145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:06.654923 1145 hybrid_clock.cc:648] HybridClock initialized: now 1786293246654910 us; error 34 us; skew 500 ppm
Aug 09 16:34:06 dist-test-slave-j2vl krb5kdc[1113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293246, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:06.657630 1145 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:34:06.658175 1145 webserver.cc:460] Webserver started at http://127.0.0.1:35739/ using document root <none> and password file <none>
I20260809 16:34:06.658526 1145 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:06.658630 1145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:06.658823 1145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:06.659955 1145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "ef1f6131d0ac461a8048128d8ea22982"
format_stamp: "Formatted at 2026-08-09 16:34:06 on dist-test-slave-j2vl"
server_key: "1915186188c7257244d3ce03d678e8f8"
server_key_iv: "3a7657ecfe6b1fd0e2c2476b7f3dc3ea"
server_key_version: "encryptionkey@0"
I20260809 16:34:06.660332 1145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "ef1f6131d0ac461a8048128d8ea22982"
format_stamp: "Formatted at 2026-08-09 16:34:06 on dist-test-slave-j2vl"
server_key: "1915186188c7257244d3ce03d678e8f8"
server_key_iv: "3a7657ecfe6b1fd0e2c2476b7f3dc3ea"
server_key_version: "encryptionkey@0"
I20260809 16:34:06.661839 1145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:34:06.662606 1173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:06.662843 1145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:06.662969 1145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "ef1f6131d0ac461a8048128d8ea22982"
format_stamp: "Formatted at 2026-08-09 16:34:06 on dist-test-slave-j2vl"
server_key: "1915186188c7257244d3ce03d678e8f8"
server_key_iv: "3a7657ecfe6b1fd0e2c2476b7f3dc3ea"
server_key_version: "encryptionkey@0"
I20260809 16:34:06.663054 1145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:06.677843 1145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:06.680437 1145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:06.680630 1145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:06.685153 1145 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42905
I20260809 16:34:06.685187 1265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42905 every 8 connection(s)
I20260809 16:34:06.685629 1145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:06.686568 1267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:06.688956 1267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982: Bootstrap starting.
I20260809 16:34:06.689579 1267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:06.689831 1267 log.cc:824] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:06.690595 1267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982: No bootstrap required, opened a new log
I20260809 16:34:06.692525 1267 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1f6131d0ac461a8048128d8ea22982" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42905 } }
I20260809 16:34:06.692687 1267 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:06.692710 1267 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef1f6131d0ac461a8048128d8ea22982, State: Initialized, Role: FOLLOWER
I20260809 16:34:06.692826 1267 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1f6131d0ac461a8048128d8ea22982" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42905 } }
I20260809 16:34:06.692896 1267 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:06.692920 1267 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:06.692955 1267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:06.693454 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 1145
I20260809 16:34:06.693598 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:34:06.693714 1267 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1f6131d0ac461a8048128d8ea22982" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42905 } }
I20260809 16:34:06.693822 30814 external_mini_cluster.cc:1428] Setting key 333f324ba2ed0f586ef9e429fc52c2d2
I20260809 16:34:06.693923 1267 leader_election.cc:302] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef1f6131d0ac461a8048128d8ea22982; no voters:
I20260809 16:34:06.694177 1267 leader_election.cc:288] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:06.694236 1273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:06.694466 1273 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [term 1 LEADER]: Becoming Leader. State: Replica: ef1f6131d0ac461a8048128d8ea22982, State: Running, Role: LEADER
I20260809 16:34:06.694583 1273 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1f6131d0ac461a8048128d8ea22982" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42905 } }
I20260809 16:34:06.694639 1267 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:06.694991 1273 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef1f6131d0ac461a8048128d8ea22982" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1f6131d0ac461a8048128d8ea22982" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42905 } } }
I20260809 16:34:06.695076 1273 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:06.695212 1273 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef1f6131d0ac461a8048128d8ea22982. Latest consensus state: current_term: 1 leader_uuid: "ef1f6131d0ac461a8048128d8ea22982" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1f6131d0ac461a8048128d8ea22982" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42905 } } }
I20260809 16:34:06.695271 1273 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:06.695436 1294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:06.696269 1294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:06.698555 1294 catalog_manager.cc:1347] Generated new cluster ID: 852d6041d0484fca83a47045b825f872
I20260809 16:34:06.698626 1294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:06.701802 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:06.694570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:39731 (local address 127.0.0.1:42905)
0809 16:34:06.694797 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:06.694807 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:06.696406 (+ 1599us) server_negotiation.cc:407] Connection header received
0809 16:34:06.696704 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:06.696711 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:06.696783 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:06.696900 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:06.697295 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:06.698438 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:06.699560 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:06.700164 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:06.700267 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:06.701076 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:06.701080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:06.701080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:06.701287 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:06.701331 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:06.701368 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:06.701450 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:06.701562 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 16:34:06.701588 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
I20260809 16:34:06.702876 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:06.713552 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:06.704616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:60503 (local address 127.0.0.1:42905)
0809 16:34:06.704660 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:06.704662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:06.709591 (+ 4929us) server_negotiation.cc:407] Connection header received
0809 16:34:06.709663 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:06.709664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:06.709678 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:06.709742 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:06.709915 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:06.710679 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:06.711763 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:06.712200 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:06.712285 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:06.713155 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:06.713156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:06.713157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:06.713162 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:06.713179 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:06.713183 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:06.713240 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:06.713362 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 16:34:06.713378 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:34:06.722877 1294 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:06.723488 1294 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:06.727483 1294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef1f6131d0ac461a8048128d8ea22982: Generated new TSK 0
I20260809 16:34:06.727705 1294 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:06.736164 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 1145
W20260809 16:34:06.750928 1111 connection.cc:535] client connection to 127.0.0.1:42905 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42905): Connection reset by peer (error 104)
2026-08-09T16:34:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1630 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:06 dist-test-slave-j2vl krb5kdc[1323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:06 dist-test-slave-j2vl krb5kdc[1323](info): set up 2 sockets
Aug 09 16:34:06 dist-test-slave-j2vl krb5kdc[1323](info): commencing operation
krb5kdc: starting...
W20260809 16:34:08.054330 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:34:08 dist-test-slave-j2vl krb5kdc[1323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:34:08.194912 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:08.280352 1340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:08.280572 1340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:08.280598 1340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:08.281761 1340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:08.281806 1340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:08.281821 1340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:08.281834 1340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:08.281847 1340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:08.283345 1340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35087
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:08.283555 1340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:08.283746 1340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:08.289222 1351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:08.289270 1348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:08.289222 1349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:08.289273 1340 server_base.cc:1034] running on GCE node
I20260809 16:34:08.289817 1340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:08.290135 1340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:08.291280 1340 hybrid_clock.cc:648] HybridClock initialized: now 1786293248291220 us; error 74 us; skew 500 ppm
Aug 09 16:34:08 dist-test-slave-j2vl krb5kdc[1323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293248, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:08.293895 1340 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:34:08.294379 1340 webserver.cc:460] Webserver started at http://127.0.0.1:34125/ using document root <none> and password file <none>
I20260809 16:34:08.294636 1340 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:08.294716 1340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:08.294873 1340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:08.295956 1340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "1fac1e7b466c4698940985cb0d4625f6"
format_stamp: "Formatted at 2026-08-09 16:34:08 on dist-test-slave-j2vl"
server_key: "fbb039849615a912b642470243d2e552"
server_key_iv: "2498bf184aed2897e26c410f55a65907"
server_key_version: "encryptionkey@0"
I20260809 16:34:08.296326 1340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "1fac1e7b466c4698940985cb0d4625f6"
format_stamp: "Formatted at 2026-08-09 16:34:08 on dist-test-slave-j2vl"
server_key: "fbb039849615a912b642470243d2e552"
server_key_iv: "2498bf184aed2897e26c410f55a65907"
server_key_version: "encryptionkey@0"
I20260809 16:34:08.297775 1340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:34:08.298547 1367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:08.298732 1340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:34:08.298849 1340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "1fac1e7b466c4698940985cb0d4625f6"
format_stamp: "Formatted at 2026-08-09 16:34:08 on dist-test-slave-j2vl"
server_key: "fbb039849615a912b642470243d2e552"
server_key_iv: "2498bf184aed2897e26c410f55a65907"
server_key_version: "encryptionkey@0"
I20260809 16:34:08.298926 1340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:08.302563 1340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:08.304620 1340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:08.304785 1340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:08.309099 1340 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35087
I20260809 16:34:08.309172 1443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35087 every 8 connection(s)
I20260809 16:34:08.309597 1340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:08.310514 1445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:08.310906 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 1340
I20260809 16:34:08.311051 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:34:08.311234 30814 external_mini_cluster.cc:1428] Setting key d19a13aebc3f83389c686d2869f8cf78
I20260809 16:34:08.313946 1445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6: Bootstrap starting.
I20260809 16:34:08.314014 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:08.311838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:57819 (local address 127.0.0.1:35087)
0809 16:34:08.312078 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:08.312089 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:08.312125 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:34:08.312675 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:08.312683 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:08.312717 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:08.312886 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:08.313550 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:08.313554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:08.313554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:08.313559 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:08.313605 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:08.313619 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:08.313703 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:08.313767 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 16:34:08.313813 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
I20260809 16:34:08.314714 1445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:08.315069 1445 log.cc:824] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:08.315125 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:08.315819 1445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6: No bootstrap required, opened a new log
I20260809 16:34:08.317739 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:08.316719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:49395 (local address 127.0.0.1:35087)
0809 16:34:08.316760 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:08.316761 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:08.316863 (+ 102us) server_negotiation.cc:407] Connection header received
0809 16:34:08.316887 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:08.316888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:08.316892 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:08.316969 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:08.317539 (+ 570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:08.317540 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:08.317541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:08.317545 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:08.317561 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:08.317566 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:08.317627 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:08.317661 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 16:34:08.317676 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:34:08.318570 1445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fac1e7b466c4698940985cb0d4625f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35087 } }
I20260809 16:34:08.318713 1445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:08.318735 1445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fac1e7b466c4698940985cb0d4625f6, State: Initialized, Role: FOLLOWER
I20260809 16:34:08.318890 1445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fac1e7b466c4698940985cb0d4625f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35087 } }
I20260809 16:34:08.318988 1445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:08.319012 1445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:08.319042 1445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:08.319597 1445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fac1e7b466c4698940985cb0d4625f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35087 } }
I20260809 16:34:08.319686 1445 leader_election.cc:302] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fac1e7b466c4698940985cb0d4625f6; no voters:
I20260809 16:34:08.319820 1445 leader_election.cc:288] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:08.319886 1460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:08.320066 1445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:08.320082 1460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [term 1 LEADER]: Becoming Leader. State: Replica: 1fac1e7b466c4698940985cb0d4625f6, State: Running, Role: LEADER
I20260809 16:34:08.320191 1460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fac1e7b466c4698940985cb0d4625f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35087 } }
I20260809 16:34:08.320612 1462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fac1e7b466c4698940985cb0d4625f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fac1e7b466c4698940985cb0d4625f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35087 } } }
I20260809 16:34:08.320613 1463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fac1e7b466c4698940985cb0d4625f6. Latest consensus state: current_term: 1 leader_uuid: "1fac1e7b466c4698940985cb0d4625f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fac1e7b466c4698940985cb0d4625f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35087 } } }
I20260809 16:34:08.320813 1463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:08.321252 1462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:08.321393 1473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:08.321949 1473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:08.323490 1473 catalog_manager.cc:1347] Generated new cluster ID: 6eae04c5bc654c4ea5fc50b5dbb95998
I20260809 16:34:08.323553 1473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:08.341424 1473 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:08.341639 1473 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:08.345281 1473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fac1e7b466c4698940985cb0d4625f6: Generated new TSK 0
I20260809 16:34:08.345542 1473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:08.350771 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 1340
2026-08-09T16:34:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1630 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:08 dist-test-slave-j2vl krb5kdc[1495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:08 dist-test-slave-j2vl krb5kdc[1495](info): set up 2 sockets
Aug 09 16:34:08 dist-test-slave-j2vl krb5kdc[1495](info): commencing operation
krb5kdc: starting...
W20260809 16:34:09.695549 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:34:09 dist-test-slave-j2vl krb5kdc[1495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:34:09.838639 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:46825
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:38749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:46825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:09.927878 1517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:09.928180 1517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:09.928228 1517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:09.929561 1517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:09.929622 1517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:09.929649 1517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:09.929683 1517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:09.929708 1517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:09.931253 1517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:38749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:46825
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:46825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:09.931552 1517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:09.931792 1517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:09.936995 1525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:09.936975 1527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:09.937023 1517 server_base.cc:1034] running on GCE node
W20260809 16:34:09.936967 1529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:09.937426 1517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:09.937736 1517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:09.938941 1517 hybrid_clock.cc:648] HybridClock initialized: now 1786293249938892 us; error 60 us; skew 500 ppm
Aug 09 16:34:09 dist-test-slave-j2vl krb5kdc[1495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293249, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:09.941668 1517 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:34:09.942138 1517 webserver.cc:460] Webserver started at http://127.30.23.190:39027/ using document root <none> and password file <none>
I20260809 16:34:09.942327 1517 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:09.942370 1517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:09.942469 1517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:09.943496 1517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "c4e6c3f4fe204f53a748e792e3e6c058"
format_stamp: "Formatted at 2026-08-09 16:34:09 on dist-test-slave-j2vl"
server_key: "0eea8c5b074831e8f0080654aff95bfa"
server_key_iv: "39e195ea0b0f527b492be444d0d05e00"
server_key_version: "encryptionkey@0"
I20260809 16:34:09.943797 1517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "c4e6c3f4fe204f53a748e792e3e6c058"
format_stamp: "Formatted at 2026-08-09 16:34:09 on dist-test-slave-j2vl"
server_key: "0eea8c5b074831e8f0080654aff95bfa"
server_key_iv: "39e195ea0b0f527b492be444d0d05e00"
server_key_version: "encryptionkey@0"
I20260809 16:34:09.945127 1517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:34:09.945940 1542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:09.946204 1517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:34:09.946329 1517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "c4e6c3f4fe204f53a748e792e3e6c058"
format_stamp: "Formatted at 2026-08-09 16:34:09 on dist-test-slave-j2vl"
server_key: "0eea8c5b074831e8f0080654aff95bfa"
server_key_iv: "39e195ea0b0f527b492be444d0d05e00"
server_key_version: "encryptionkey@0"
I20260809 16:34:09.946415 1517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:09.960690 1517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:09.962826 1517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:09.962993 1517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:09.967026 1517 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:46825
I20260809 16:34:09.967088 1631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:46825 every 8 connection(s)
I20260809 16:34:09.967449 1517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:09.968360 1633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:09.970659 1633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058: Bootstrap starting.
I20260809 16:34:09.971194 1633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:09.971431 1633 log.cc:824] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:09.972107 1633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058: No bootstrap required, opened a new log
I20260809 16:34:09.973722 1633 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e6c3f4fe204f53a748e792e3e6c058" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 46825 } }
I20260809 16:34:09.973860 1633 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:09.973882 1633 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4e6c3f4fe204f53a748e792e3e6c058, State: Initialized, Role: FOLLOWER
I20260809 16:34:09.974015 1633 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e6c3f4fe204f53a748e792e3e6c058" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 46825 } }
I20260809 16:34:09.974092 1633 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:09.974115 1633 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:09.974143 1633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:09.974975 1633 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e6c3f4fe204f53a748e792e3e6c058" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 46825 } }
I20260809 16:34:09.975064 1633 leader_election.cc:302] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4e6c3f4fe204f53a748e792e3e6c058; no voters:
I20260809 16:34:09.975217 1633 leader_election.cc:288] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:09.975242 1642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:09.975354 1642 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [term 1 LEADER]: Becoming Leader. State: Replica: c4e6c3f4fe204f53a748e792e3e6c058, State: Running, Role: LEADER
I20260809 16:34:09.975440 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 1517
I20260809 16:34:09.975509 1642 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e6c3f4fe204f53a748e792e3e6c058" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 46825 } }
I20260809 16:34:09.975575 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:34:09.975762 30814 external_mini_cluster.cc:1428] Setting key 24c0a6712d621bc2da222c7e85d371d0
I20260809 16:34:09.975497 1633 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:09.975999 1644 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4e6c3f4fe204f53a748e792e3e6c058. Latest consensus state: current_term: 1 leader_uuid: "c4e6c3f4fe204f53a748e792e3e6c058" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e6c3f4fe204f53a748e792e3e6c058" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 46825 } } }
I20260809 16:34:09.976096 1644 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:09.976320 1643 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4e6c3f4fe204f53a748e792e3e6c058" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e6c3f4fe204f53a748e792e3e6c058" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 46825 } } }
I20260809 16:34:09.976449 1643 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:09.976320 1647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:09.977052 1647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:09.979123 1647 catalog_manager.cc:1347] Generated new cluster ID: 38de424254924f13b59c8614c9f7aa35
I20260809 16:34:09.979187 1647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:34:09 dist-test-slave-j2vl krb5kdc[1495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:34:09.988344 1647 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:09.988606 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:09.976466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38147 (local address 127.30.23.190:46825)
0809 16:34:09.977013 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:09.977025 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:09.977294 (+ 269us) server_negotiation.cc:407] Connection header received
0809 16:34:09.977449 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:09.977459 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:09.977528 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:09.977673 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:09.978537 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:09.979923 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:09.980970 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:09.981631 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:09.981749 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:09.984718 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:09.984723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:09.984723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:09.984737 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:09.986781 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:09.987266 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:09.987269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:09.987270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:09.987379 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:09.987640 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:09.987641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:09.987641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:09.987920 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:09.988112 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:09.988226 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 16:34:09.988334 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":80,"threads_started":1}
I20260809 16:34:09.989104 1647 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:09.989617 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:09.996464 1647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4e6c3f4fe204f53a748e792e3e6c058: Generated new TSK 0
I20260809 16:34:09.996657 1647 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:09.997586 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:09.990964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34315 (local address 127.30.23.190:46825)
0809 16:34:09.991004 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:09.991006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:09.991126 (+ 120us) server_negotiation.cc:407] Connection header received
0809 16:34:09.991201 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:09.991202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:09.991212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:09.991267 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:09.991772 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:09.992295 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:09.993020 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:09.993251 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:09.993327 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:09.994518 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:09.994520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:09.994520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:09.994527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:09.996016 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:09.996750 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:09.996751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:09.996752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:09.996813 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:09.997066 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:09.997067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:09.997067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:09.997168 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:09.997247 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:09.997456 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 16:34:09.997507 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:34:09.999925 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 1517
W20260809 16:34:10.026458 1491 connection.cc:535] client connection to 127.30.23.190:46825 recv error: Network error: failed to read from TLS socket (remote: 127.30.23.190:46825): Connection reset by peer (error 104)
2026-08-09T16:34:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1645 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:10 dist-test-slave-j2vl krb5kdc[1682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:10 dist-test-slave-j2vl krb5kdc[1682](info): set up 2 sockets
Aug 09 16:34:10 dist-test-slave-j2vl krb5kdc[1682](info): commencing operation
krb5kdc: starting...
W20260809 16:34:11.342604 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:34:11 dist-test-slave-j2vl krb5kdc[1682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:34:11.490136 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:36527
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:43803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:36527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:11.579715 1708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:11.579991 1708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:11.580035 1708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:11.581274 1708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:11.581348 1708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:11.581394 1708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:11.581426 1708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:11.581451 1708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:11.583005 1708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:43803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:36527
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:36527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:11.583314 1708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:11.583559 1708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:11.588605 1714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:11.588605 1718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:11.588941 1708 server_base.cc:1034] running on GCE node
W20260809 16:34:11.588621 1716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:11.589308 1708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:11.589685 1708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:11.590859 1708 hybrid_clock.cc:648] HybridClock initialized: now 1786293251590847 us; error 35 us; skew 500 ppm
Aug 09 16:34:11 dist-test-slave-j2vl krb5kdc[1682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293251, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:11.593582 1708 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:34:11.594069 1708 webserver.cc:460] Webserver started at http://127.30.23.190:42219/ using document root <none> and password file <none>
I20260809 16:34:11.594303 1708 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:11.594352 1708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:11.594481 1708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:11.595602 1708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "7d4180c5c50d4c6c94e9691615a08fba"
format_stamp: "Formatted at 2026-08-09 16:34:11 on dist-test-slave-j2vl"
server_key: "28d33a0ffd94dca3a6ee7543d5db764b"
server_key_iv: "1943dba7826cc1046bfd12fa1ab2197f"
server_key_version: "encryptionkey@0"
I20260809 16:34:11.595983 1708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "7d4180c5c50d4c6c94e9691615a08fba"
format_stamp: "Formatted at 2026-08-09 16:34:11 on dist-test-slave-j2vl"
server_key: "28d33a0ffd94dca3a6ee7543d5db764b"
server_key_iv: "1943dba7826cc1046bfd12fa1ab2197f"
server_key_version: "encryptionkey@0"
I20260809 16:34:11.597532 1708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 16:34:11.598527 1733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:11.599041 1708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:11.599143 1708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "7d4180c5c50d4c6c94e9691615a08fba"
format_stamp: "Formatted at 2026-08-09 16:34:11 on dist-test-slave-j2vl"
server_key: "28d33a0ffd94dca3a6ee7543d5db764b"
server_key_iv: "1943dba7826cc1046bfd12fa1ab2197f"
server_key_version: "encryptionkey@0"
I20260809 16:34:11.599253 1708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:11.615624 1708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:11.618434 1708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:11.618616 1708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:11.623443 1708 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:36527
I20260809 16:34:11.623487 1819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:36527 every 8 connection(s)
I20260809 16:34:11.623899 1708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:11.624871 1822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:11.626523 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 1708
I20260809 16:34:11.626690 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:34:11.626911 30814 external_mini_cluster.cc:1428] Setting key 02f91025d7bef6898cc45f69fff15c61
I20260809 16:34:11.628074 1822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba: Bootstrap starting.
I20260809 16:34:11.628868 1822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:11.629209 1822 log.cc:824] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:11.630040 1822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba: No bootstrap required, opened a new log
I20260809 16:34:11.632321 1822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4180c5c50d4c6c94e9691615a08fba" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36527 } }
I20260809 16:34:11.632493 1822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:11.632545 1822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d4180c5c50d4c6c94e9691615a08fba, State: Initialized, Role: FOLLOWER
I20260809 16:34:11.632714 1822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4180c5c50d4c6c94e9691615a08fba" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36527 } }
I20260809 16:34:11.632807 1822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:11.632854 1822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:11.632907 1822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:11.633497 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:11.627537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38557 (local address 127.30.23.190:36527)
0809 16:34:11.627754 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:11.627765 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:11.627799 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:34:11.628354 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:11.628363 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:11.628459 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:11.628626 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:11.628860 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:11.630248 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:11.631045 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:11.631700 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:11.631848 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:11.632689 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:11.632694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:11.632694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:11.632953 (+ 259us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:11.632999 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:11.633015 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:11.633124 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:11.633212 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 16:34:11.633243 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
I20260809 16:34:11.633700 1822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4180c5c50d4c6c94e9691615a08fba" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36527 } }
I20260809 16:34:11.634001 1822 leader_election.cc:302] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d4180c5c50d4c6c94e9691615a08fba; no voters:
I20260809 16:34:11.634203 1822 leader_election.cc:288] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:11.634240 1833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:11.634353 1833 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [term 1 LEADER]: Becoming Leader. State: Replica: 7d4180c5c50d4c6c94e9691615a08fba, State: Running, Role: LEADER
I20260809 16:34:11.634507 1833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4180c5c50d4c6c94e9691615a08fba" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36527 } }
I20260809 16:34:11.634567 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:11.634599 1822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:11.634945 1833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d4180c5c50d4c6c94e9691615a08fba. Latest consensus state: current_term: 1 leader_uuid: "7d4180c5c50d4c6c94e9691615a08fba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4180c5c50d4c6c94e9691615a08fba" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36527 } } }
I20260809 16:34:11.635020 1833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:11.635519 1839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:11.636106 1839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:11.636375 1836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d4180c5c50d4c6c94e9691615a08fba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4180c5c50d4c6c94e9691615a08fba" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 36527 } } }
I20260809 16:34:11.636588 1836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:11.637856 1839 catalog_manager.cc:1347] Generated new cluster ID: 86ee2cb31a5c49e78354d64d4a32a010
I20260809 16:34:11.637912 1839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:11.640245 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:11.636259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54883 (local address 127.30.23.190:36527)
0809 16:34:11.636302 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:11.636304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:11.636574 (+ 270us) server_negotiation.cc:407] Connection header received
0809 16:34:11.636647 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:11.636648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:11.636659 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:11.636711 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:11.636934 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:11.637690 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:11.638730 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:11.639013 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:11.639079 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:11.639920 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:11.639921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:11.639922 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:11.639928 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:11.639943 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:11.639947 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:11.639997 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:11.640160 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 16:34:11.640171 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:34:11.649246 1839 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:11.649876 1839 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:11.656643 1839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d4180c5c50d4c6c94e9691615a08fba: Generated new TSK 0
I20260809 16:34:11.656898 1839 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:11.664634 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 1708
W20260809 16:34:11.679247 1680 connection.cc:535] client connection to 127.30.23.190:36527 recv error: Network error: failed to read from TLS socket (remote: 127.30.23.190:36527): Connection reset by peer (error 104)
2026-08-09T16:34:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1652 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:11 dist-test-slave-j2vl krb5kdc[1878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:11 dist-test-slave-j2vl krb5kdc[1878](info): set up 2 sockets
Aug 09 16:34:11 dist-test-slave-j2vl krb5kdc[1878](info): commencing operation
krb5kdc: starting...
W20260809 16:34:13.016633 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:34:13 dist-test-slave-j2vl krb5kdc[1878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:34:13.148337 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:32883
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:32883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:13.234824 1898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:13.235060 1898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:13.235096 1898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:13.236224 1898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:13.236295 1898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:13.236320 1898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:13.236348 1898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:13.236378 1898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:13.237882 1898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:32883
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:32883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:13.238170 1898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:13.238406 1898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:13.243876 1910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:13.243978 1898 server_base.cc:1034] running on GCE node
W20260809 16:34:13.243876 1907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:13.243944 1905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:13.244504 1898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:13.244805 1898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:13.245975 1898 hybrid_clock.cc:648] HybridClock initialized: now 1786293253245952 us; error 35 us; skew 500 ppm
Aug 09 16:34:13 dist-test-slave-j2vl krb5kdc[1878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293253, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:13.248737 1898 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:34:13.249234 1898 webserver.cc:460] Webserver started at http://127.30.23.190:33011/ using document root <none> and password file <none>
I20260809 16:34:13.249503 1898 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:13.249552 1898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:13.249698 1898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:13.250806 1898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "d676e81a997040fb9e83e2c3569c29c1"
format_stamp: "Formatted at 2026-08-09 16:34:13 on dist-test-slave-j2vl"
server_key: "f1c1130889e1a628bc136a6d3887bac9"
server_key_iv: "392657a0d97c2e92c34aae06ef638a6d"
server_key_version: "encryptionkey@0"
I20260809 16:34:13.251202 1898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "d676e81a997040fb9e83e2c3569c29c1"
format_stamp: "Formatted at 2026-08-09 16:34:13 on dist-test-slave-j2vl"
server_key: "f1c1130889e1a628bc136a6d3887bac9"
server_key_iv: "392657a0d97c2e92c34aae06ef638a6d"
server_key_version: "encryptionkey@0"
I20260809 16:34:13.252614 1898 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:13.253499 1923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:13.253720 1898 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:13.253837 1898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "d676e81a997040fb9e83e2c3569c29c1"
format_stamp: "Formatted at 2026-08-09 16:34:13 on dist-test-slave-j2vl"
server_key: "f1c1130889e1a628bc136a6d3887bac9"
server_key_iv: "392657a0d97c2e92c34aae06ef638a6d"
server_key_version: "encryptionkey@0"
I20260809 16:34:13.253937 1898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:13.257396 1898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:13.259831 1898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:13.259989 1898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:13.264299 1898 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:32883
I20260809 16:34:13.264335 2012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:32883 every 8 connection(s)
I20260809 16:34:13.264701 1898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:13.265859 2013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:13.268249 2013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1: Bootstrap starting.
I20260809 16:34:13.268872 2013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:13.269169 2013 log.cc:824] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:13.270018 2013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1: No bootstrap required, opened a new log
I20260809 16:34:13.271672 2013 raft_consensus.cc:348] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d676e81a997040fb9e83e2c3569c29c1" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 32883 } }
I20260809 16:34:13.271814 2013 raft_consensus.cc:374] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:13.271838 2013 raft_consensus.cc:729] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d676e81a997040fb9e83e2c3569c29c1, State: Initialized, Role: FOLLOWER
I20260809 16:34:13.272010 2013 consensus_queue.cc:260] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d676e81a997040fb9e83e2c3569c29c1" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 32883 } }
I20260809 16:34:13.272076 2013 raft_consensus.cc:388] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:13.272122 2013 raft_consensus.cc:482] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:13.272179 2013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:13.273085 2013 raft_consensus.cc:504] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d676e81a997040fb9e83e2c3569c29c1" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 32883 } }
I20260809 16:34:13.273188 2013 leader_election.cc:302] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d676e81a997040fb9e83e2c3569c29c1; no voters:
I20260809 16:34:13.273420 2013 leader_election.cc:288] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:13.273614 2020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:13.273717 2013 sys_catalog.cc:564] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:13.273828 2020 raft_consensus.cc:686] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [term 1 LEADER]: Becoming Leader. State: Replica: d676e81a997040fb9e83e2c3569c29c1, State: Running, Role: LEADER
I20260809 16:34:13.273947 2020 consensus_queue.cc:237] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d676e81a997040fb9e83e2c3569c29c1" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 32883 } }
I20260809 16:34:13.274240 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 1898
I20260809 16:34:13.274380 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:34:13.274591 30814 external_mini_cluster.cc:1428] Setting key dbeb3922a3cb8c029639404712ad90e3
I20260809 16:34:13.274586 2023 sys_catalog.cc:455] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d676e81a997040fb9e83e2c3569c29c1. Latest consensus state: current_term: 1 leader_uuid: "d676e81a997040fb9e83e2c3569c29c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d676e81a997040fb9e83e2c3569c29c1" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 32883 } } }
I20260809 16:34:13.274695 2023 sys_catalog.cc:458] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:13.276101 2022 sys_catalog.cc:455] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d676e81a997040fb9e83e2c3569c29c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d676e81a997040fb9e83e2c3569c29c1" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 32883 } } }
W20260809 16:34:13.276167 2052 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 16:34:13.276274 2022 sys_catalog.cc:458] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1 [sys.catalog]: This master's current role is: LEADER
W20260809 16:34:13.276436 2052 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:34:13.276857 2037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:13.277105 2037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:13.277534 2053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:13.275686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34427 (local address 127.30.23.190:32883)
0809 16:34:13.275891 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:13.275901 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:13.275936 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:34:13.276078 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:13.276087 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:13.276122 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:13.276587 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:13.276978 (+ 391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:13.276982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:13.276982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:13.276986 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:13.277027 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:13.277044 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:13.277118 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:13.277281 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 16:34:13.277324 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260809 16:34:13.278532 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:13.278834 2037 catalog_manager.cc:1347] Generated new cluster ID: 6a52b27b32a647ab8f36287be62a752f
I20260809 16:34:13.278870 2037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:13.281241 2053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:13.280119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35499 (local address 127.30.23.190:32883)
0809 16:34:13.280158 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:13.280160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:13.280230 (+ 70us) server_negotiation.cc:407] Connection header received
0809 16:34:13.280322 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:13.280323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:13.280328 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:13.280383 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:13.281016 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:13.281018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:13.281018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:13.281022 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:13.281040 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:13.281045 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:13.281110 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:13.281159 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 16:34:13.281173 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:34:13.294572 2037 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:13.294806 2037 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:13.297861 2037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d676e81a997040fb9e83e2c3569c29c1: Generated new TSK 0
I20260809 16:34:13.298038 2037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:13.305976 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 1898
2026-08-09T16:34:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1637 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:13 dist-test-slave-j2vl krb5kdc[2067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:13 dist-test-slave-j2vl krb5kdc[2067](info): set up 2 sockets
Aug 09 16:34:13 dist-test-slave-j2vl krb5kdc[2067](info): commencing operation
krb5kdc: starting...
W20260809 16:34:14.662837 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:34:14 dist-test-slave-j2vl krb5kdc[2067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:34:14.809540 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:35363
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:45259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:35363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:14.896150 2084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:14.896440 2084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:14.896488 2084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:14.897742 2084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:14.897871 2084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:14.897938 2084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:14.897969 2084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:14.898000 2084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:14.899631 2084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:45259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:35363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:35363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:14.899958 2084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:14.900208 2084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:14.905299 2095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:14.905328 2093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:14.905508 2084 server_base.cc:1034] running on GCE node
W20260809 16:34:14.905325 2097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:14.905896 2084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:14.906239 2084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:14.907450 2084 hybrid_clock.cc:648] HybridClock initialized: now 1786293254907428 us; error 41 us; skew 500 ppm
Aug 09 16:34:14 dist-test-slave-j2vl krb5kdc[2067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293254, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:14.910153 2084 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:34:14.910696 2084 webserver.cc:460] Webserver started at http://127.30.23.190:44215/ using document root <none> and password file <none>
I20260809 16:34:14.910964 2084 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:14.911037 2084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:14.911197 2084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:14.912338 2084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "81196fe0ff1847f1be4014bbec044e3f"
format_stamp: "Formatted at 2026-08-09 16:34:14 on dist-test-slave-j2vl"
server_key: "3f35ac58c62c72c37e5356f35f3da6f6"
server_key_iv: "aece85e6e6a8f8f0a0c64695e4281675"
server_key_version: "encryptionkey@0"
I20260809 16:34:14.912737 2084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "81196fe0ff1847f1be4014bbec044e3f"
format_stamp: "Formatted at 2026-08-09 16:34:14 on dist-test-slave-j2vl"
server_key: "3f35ac58c62c72c37e5356f35f3da6f6"
server_key_iv: "aece85e6e6a8f8f0a0c64695e4281675"
server_key_version: "encryptionkey@0"
I20260809 16:34:14.914315 2084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:34:14.915213 2106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:14.915491 2084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:34:14.915625 2084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "81196fe0ff1847f1be4014bbec044e3f"
format_stamp: "Formatted at 2026-08-09 16:34:14 on dist-test-slave-j2vl"
server_key: "3f35ac58c62c72c37e5356f35f3da6f6"
server_key_iv: "aece85e6e6a8f8f0a0c64695e4281675"
server_key_version: "encryptionkey@0"
I20260809 16:34:14.915719 2084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:14.934760 2084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:14.937551 2084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:14.937772 2084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:14.942402 2084 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:35363
I20260809 16:34:14.942435 2202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:35363 every 8 connection(s)
I20260809 16:34:14.942843 2084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:14.943871 2204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:14.946317 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 2084
I20260809 16:34:14.946496 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:34:14.946730 30814 external_mini_cluster.cc:1428] Setting key 151f8672ec0658e954797cd975178cdc
I20260809 16:34:14.946805 2204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f: Bootstrap starting.
I20260809 16:34:14.947678 2204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:14.947964 2204 log.cc:824] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:14.948817 2204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f: No bootstrap required, opened a new log
I20260809 16:34:14.951149 2204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81196fe0ff1847f1be4014bbec044e3f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35363 } }
I20260809 16:34:14.951328 2204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:14.951359 2204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81196fe0ff1847f1be4014bbec044e3f, State: Initialized, Role: FOLLOWER
I20260809 16:34:14.951493 2204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81196fe0ff1847f1be4014bbec044e3f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35363 } }
I20260809 16:34:14.951596 2204 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:14.951656 2204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:14.951707 2204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:14.952354 2204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81196fe0ff1847f1be4014bbec044e3f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35363 } }
I20260809 16:34:14.952490 2204 leader_election.cc:302] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81196fe0ff1847f1be4014bbec044e3f; no voters:
I20260809 16:34:14.952692 2204 leader_election.cc:288] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:14.952764 2217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:14.952997 2217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [term 1 LEADER]: Becoming Leader. State: Replica: 81196fe0ff1847f1be4014bbec044e3f, State: Running, Role: LEADER
I20260809 16:34:14.953027 2204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:14.953101 2217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81196fe0ff1847f1be4014bbec044e3f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35363 } }
I20260809 16:34:14.953436 2217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81196fe0ff1847f1be4014bbec044e3f. Latest consensus state: current_term: 1 leader_uuid: "81196fe0ff1847f1be4014bbec044e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81196fe0ff1847f1be4014bbec044e3f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35363 } } }
I20260809 16:34:14.953542 2217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:14.953447 2218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81196fe0ff1847f1be4014bbec044e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81196fe0ff1847f1be4014bbec044e3f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 35363 } } }
I20260809 16:34:14.953683 2218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:14.953783 2227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:14.954310 2227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:34:14 dist-test-slave-j2vl krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:34:14.956969 2227 catalog_manager.cc:1347] Generated new cluster ID: 5215dd4d07a44d80a9306e0b82d39002
I20260809 16:34:14.957046 2227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:14.959903 2212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:14.947478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53147 (local address 127.30.23.190:35363)
0809 16:34:14.947673 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:14.947683 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:14.947720 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:34:14.948413 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:14.948434 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:14.948502 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:14.948635 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:14.949554 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:14.950483 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:14.951625 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:14.952137 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:14.952252 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:14.955793 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:14.955797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:14.955797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:14.955812 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:14.958102 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:14.958564 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:14.958567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:14.958567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:14.958685 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:14.958900 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:14.958901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:14.958901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:14.959239 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:14.959400 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:14.959515 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 16:34:14.959674 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260809 16:34:14.960939 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:14.967832 2227 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:14.968678 2227 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:14.970933 2212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:14.962808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48037 (local address 127.30.23.190:35363)
0809 16:34:14.962877 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:14.962879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:14.963086 (+ 207us) server_negotiation.cc:407] Connection header received
0809 16:34:14.963153 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:14.963153 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:14.963165 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:14.963222 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:14.963928 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:14.964677 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:14.965868 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:14.966194 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:14.966254 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:14.967893 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:14.967895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:14.967895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:14.967903 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:14.969671 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:14.970102 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:14.970103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:14.970104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:14.970166 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:14.970379 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:14.970380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:14.970380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:14.970503 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:14.970594 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:14.970742 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 16:34:14.970840 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 16:34:14.981259 2227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81196fe0ff1847f1be4014bbec044e3f: Generated new TSK 0
I20260809 16:34:14.981583 2227 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:14.987263 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 2084
2026-08-09T16:34:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1690 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:15 dist-test-slave-j2vl krb5kdc[2263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:15 dist-test-slave-j2vl krb5kdc[2263](info): set up 2 sockets
Aug 09 16:34:15 dist-test-slave-j2vl krb5kdc[2263](info): commencing operation
krb5kdc: starting...
W20260809 16:34:16.295863 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:34:16 dist-test-slave-j2vl krb5kdc[2263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:34:16.429502 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:40795
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:44435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:40795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:16.514928 2283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:16.515185 2283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:16.515228 2283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:16.516417 2283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:16.516491 2283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:16.516520 2283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:16.516552 2283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:16.516582 2283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:16.518105 2283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:44435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:40795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:40795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:16.518396 2283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:16.518636 2283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:16.523888 2292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:16.523898 2291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:16.524011 2283 server_base.cc:1034] running on GCE node
W20260809 16:34:16.523905 2294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:16.524399 2283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:16.524708 2283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:16.525884 2283 hybrid_clock.cc:648] HybridClock initialized: now 1786293256525886 us; error 69 us; skew 500 ppm
Aug 09 16:34:16 dist-test-slave-j2vl krb5kdc[2263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293256, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:16.531327 2283 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:34:16.531787 2283 webserver.cc:460] Webserver started at http://127.30.23.190:37585/ using document root <none> and password file <none>
I20260809 16:34:16.532027 2283 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:16.532101 2283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:16.532254 2283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:16.533425 2283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "f7ade2b6cebd4e12bcd8977f9d2871e7"
format_stamp: "Formatted at 2026-08-09 16:34:16 on dist-test-slave-j2vl"
server_key: "f68fd72737cee434eb1ff115d9c3852d"
server_key_iv: "a65a9e3d0306c09ed43d5f8693877375"
server_key_version: "encryptionkey@0"
I20260809 16:34:16.533792 2283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "f7ade2b6cebd4e12bcd8977f9d2871e7"
format_stamp: "Formatted at 2026-08-09 16:34:16 on dist-test-slave-j2vl"
server_key: "f68fd72737cee434eb1ff115d9c3852d"
server_key_iv: "a65a9e3d0306c09ed43d5f8693877375"
server_key_version: "encryptionkey@0"
I20260809 16:34:16.535204 2283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:34:16.535967 2304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:16.536150 2283 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:34:16.536259 2283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "f7ade2b6cebd4e12bcd8977f9d2871e7"
format_stamp: "Formatted at 2026-08-09 16:34:16 on dist-test-slave-j2vl"
server_key: "f68fd72737cee434eb1ff115d9c3852d"
server_key_iv: "a65a9e3d0306c09ed43d5f8693877375"
server_key_version: "encryptionkey@0"
I20260809 16:34:16.536348 2283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:16.562932 2283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:16.565251 2283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:16.565455 2283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:16.569785 2283 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:40795
I20260809 16:34:16.569897 2393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:40795 every 8 connection(s)
I20260809 16:34:16.570217 2283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:16.571259 2394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:16.573657 2394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7: Bootstrap starting.
I20260809 16:34:16.574249 2394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:16.574536 2394 log.cc:824] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:16.575277 2394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7: No bootstrap required, opened a new log
I20260809 16:34:16.575532 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 2283
I20260809 16:34:16.575670 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:34:16.575853 30814 external_mini_cluster.cc:1428] Setting key dca5fd0d1de4ce1ec135db3ff3e9af07
I20260809 16:34:16.577488 2394 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ade2b6cebd4e12bcd8977f9d2871e7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40795 } }
I20260809 16:34:16.577646 2394 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:16.577687 2394 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ade2b6cebd4e12bcd8977f9d2871e7, State: Initialized, Role: FOLLOWER
I20260809 16:34:16.577880 2394 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ade2b6cebd4e12bcd8977f9d2871e7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40795 } }
I20260809 16:34:16.577970 2394 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:16.577993 2394 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:16.578056 2394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:16.578584 2394 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ade2b6cebd4e12bcd8977f9d2871e7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40795 } }
I20260809 16:34:16.578671 2394 leader_election.cc:302] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7ade2b6cebd4e12bcd8977f9d2871e7; no voters:
I20260809 16:34:16.578887 2394 leader_election.cc:288] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:16.578909 2400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:16.579021 2400 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [term 1 LEADER]: Becoming Leader. State: Replica: f7ade2b6cebd4e12bcd8977f9d2871e7, State: Running, Role: LEADER
I20260809 16:34:16.579155 2394 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:16.579177 2400 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ade2b6cebd4e12bcd8977f9d2871e7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40795 } }
I20260809 16:34:16.579625 2402 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7ade2b6cebd4e12bcd8977f9d2871e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ade2b6cebd4e12bcd8977f9d2871e7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40795 } } }
I20260809 16:34:16.579880 2402 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:16.579789 2401 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7ade2b6cebd4e12bcd8977f9d2871e7. Latest consensus state: current_term: 1 leader_uuid: "f7ade2b6cebd4e12bcd8977f9d2871e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ade2b6cebd4e12bcd8977f9d2871e7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 40795 } } }
I20260809 16:34:16.580227 2401 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:16.580433 2411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:16.580993 2411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:16.582717 2399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:16.576396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38985 (local address 127.30.23.190:40795)
0809 16:34:16.576600 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:16.576607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:16.576633 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:34:16.577472 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:16.577480 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:16.577532 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:16.577676 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:16.577912 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:16.579409 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:16.580581 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:16.581141 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:16.581285 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:16.581990 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:16.581995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:16.581996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:16.582202 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:16.582242 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:16.582257 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:16.582340 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:16.582436 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 16:34:16.582499 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260809 16:34:16.583213 2411 catalog_manager.cc:1347] Generated new cluster ID: 35926967fb6643f9b30ef50ea24c63d9
I20260809 16:34:16.583315 2411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:16.583802 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:16.588318 2399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:16.585228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38089 (local address 127.30.23.190:40795)
0809 16:34:16.585271 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:16.585273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:16.585428 (+ 155us) server_negotiation.cc:407] Connection header received
0809 16:34:16.585499 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:16.585499 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:16.585510 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:16.585566 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:16.585719 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:16.586270 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:16.587056 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:16.587268 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:16.587341 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:16.588061 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:16.588062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:16.588062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:16.588067 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:16.588081 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:16.588085 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:16.588133 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:16.588191 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 16:34:16.588214 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:34:16.592773 2411 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:16.593384 2411 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:16.600864 2411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7ade2b6cebd4e12bcd8977f9d2871e7: Generated new TSK 0
I20260809 16:34:16.601073 2411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:16.605587 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 2283
2026-08-09T16:34:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1614 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:16 dist-test-slave-j2vl krb5kdc[2442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:16 dist-test-slave-j2vl krb5kdc[2442](info): set up 2 sockets
Aug 09 16:34:16 dist-test-slave-j2vl krb5kdc[2442](info): commencing operation
krb5kdc: starting...
W20260809 16:34:17.909317 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:34:18 dist-test-slave-j2vl krb5kdc[2442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:34:18.050216 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:41963
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:41659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:41963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:18.137684 2462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:18.137912 2462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:18.137936 2462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:18.139068 2462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:18.139109 2462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:18.139123 2462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:18.139138 2462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:18.139151 2462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:18.140619 2462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:41659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:41963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:41963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:18.140838 2462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:18.141029 2462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:18.146231 2473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:18.146257 2468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:18.146235 2477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:18.146656 2462 server_base.cc:1034] running on GCE node
I20260809 16:34:18.146973 2462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:18.147285 2462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:18.148510 2462 hybrid_clock.cc:648] HybridClock initialized: now 1786293258148478 us; error 47 us; skew 500 ppm
Aug 09 16:34:18 dist-test-slave-j2vl krb5kdc[2442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:18.151297 2462 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:34:18.151800 2462 webserver.cc:460] Webserver started at http://127.30.23.190:40131/ using document root <none> and password file <none>
I20260809 16:34:18.152066 2462 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:18.152139 2462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:18.152292 2462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:18.153465 2462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "87f5fd4fe10b4142a8c922330c3b05e7"
format_stamp: "Formatted at 2026-08-09 16:34:18 on dist-test-slave-j2vl"
server_key: "883ed8699c8aa26f9becab61c70a5326"
server_key_iv: "f0bbef9f195eb2919c131f0473a16259"
server_key_version: "encryptionkey@0"
I20260809 16:34:18.153855 2462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "87f5fd4fe10b4142a8c922330c3b05e7"
format_stamp: "Formatted at 2026-08-09 16:34:18 on dist-test-slave-j2vl"
server_key: "883ed8699c8aa26f9becab61c70a5326"
server_key_iv: "f0bbef9f195eb2919c131f0473a16259"
server_key_version: "encryptionkey@0"
I20260809 16:34:18.155656 2462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:34:18.156876 2486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:18.157060 2462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:34:18.157173 2462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "87f5fd4fe10b4142a8c922330c3b05e7"
format_stamp: "Formatted at 2026-08-09 16:34:18 on dist-test-slave-j2vl"
server_key: "883ed8699c8aa26f9becab61c70a5326"
server_key_iv: "f0bbef9f195eb2919c131f0473a16259"
server_key_version: "encryptionkey@0"
I20260809 16:34:18.157258 2462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:18.161166 2462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:18.164922 2462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:18.165091 2462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:18.169864 2462 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:41963
I20260809 16:34:18.169945 2571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:41963 every 8 connection(s)
I20260809 16:34:18.170291 2462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:18.171370 2572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:18.173780 2572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7: Bootstrap starting.
I20260809 16:34:18.174335 2572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:18.174590 2572 log.cc:824] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:18.175324 2572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7: No bootstrap required, opened a new log
I20260809 16:34:18.176344 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 2462
I20260809 16:34:18.176491 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:34:18.176698 30814 external_mini_cluster.cc:1428] Setting key a214f243b6a08845b1c6814bed20790c
I20260809 16:34:18.177639 2572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f5fd4fe10b4142a8c922330c3b05e7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41963 } }
I20260809 16:34:18.177814 2572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:18.177862 2572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f5fd4fe10b4142a8c922330c3b05e7, State: Initialized, Role: FOLLOWER
I20260809 16:34:18.178005 2572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f5fd4fe10b4142a8c922330c3b05e7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41963 } }
I20260809 16:34:18.178108 2572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:18.178154 2572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:18.178205 2572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:18.179021 2572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f5fd4fe10b4142a8c922330c3b05e7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41963 } }
I20260809 16:34:18.179152 2572 leader_election.cc:302] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87f5fd4fe10b4142a8c922330c3b05e7; no voters:
I20260809 16:34:18.179327 2572 leader_election.cc:288] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:18.179371 2579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:18.179564 2579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [term 1 LEADER]: Becoming Leader. State: Replica: 87f5fd4fe10b4142a8c922330c3b05e7, State: Running, Role: LEADER
I20260809 16:34:18.179698 2572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:18.179694 2579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f5fd4fe10b4142a8c922330c3b05e7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41963 } }
I20260809 16:34:18.179728 2578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:18.177493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54793 (local address 127.30.23.190:41963)
0809 16:34:18.178095 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:18.178103 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:18.178156 (+ 53us) server_negotiation.cc:407] Connection header received
0809 16:34:18.178315 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:18.178321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:18.178353 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:18.178472 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:18.179229 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:18.179234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:18.179234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:18.179239 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:18.179294 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:18.179306 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:18.179398 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:18.179475 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 16:34:18.179519 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":510,"threads_started":1}
I20260809 16:34:18.180094 2579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87f5fd4fe10b4142a8c922330c3b05e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f5fd4fe10b4142a8c922330c3b05e7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41963 } } }
I20260809 16:34:18.180254 2579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:18.180446 2579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87f5fd4fe10b4142a8c922330c3b05e7. Latest consensus state: current_term: 1 leader_uuid: "87f5fd4fe10b4142a8c922330c3b05e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f5fd4fe10b4142a8c922330c3b05e7" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 41963 } } }
I20260809 16:34:18.180580 2579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:18.180913 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:18.181712 2585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:18.182372 2585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:18.183882 2578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:18.182833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54795 (local address 127.30.23.190:41963)
0809 16:34:18.182878 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:18.182880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:18.182890 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:34:18.182913 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:18.182914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:18.182921 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:18.182983 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:18.183626 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:18.183627 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:18.183628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:18.183633 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:18.183648 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:18.183652 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:18.183684 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:18.183783 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 16:34:18.183799 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 16:34:18.184330 2585 catalog_manager.cc:1347] Generated new cluster ID: 6e836a403add41a38c088f07093289c6
I20260809 16:34:18.184382 2585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:18.191534 2585 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:18.191752 2585 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:18.195530 2585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87f5fd4fe10b4142a8c922330c3b05e7: Generated new TSK 0
I20260809 16:34:18.195739 2585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:18.201918 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 2462
2026-08-09T16:34:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1592 ms)
[----------] 18 tests from AuthTokenIssuingTest (29921 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:18 dist-test-slave-j2vl krb5kdc[2621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:18 dist-test-slave-j2vl krb5kdc[2621](info): set up 2 sockets
Aug 09 16:34:18 dist-test-slave-j2vl krb5kdc[2621](info): commencing operation
krb5kdc: starting...
W20260809 16:34:19.527987 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:34:19 dist-test-slave-j2vl krb5kdc[2621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:34:19.668628 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:34993
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:34993,127.30.23.189:38543,127.30.23.188:37731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:19.757076 2639 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:34:19.757429 2639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:19.757496 2639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:19.757545 2639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:19.758836 2639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:19.758917 2639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:19.758945 2639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:19.758977 2639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:19.759027 2639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:19.760665 2639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:34993,127.30.23.189:38543,127.30.23.188:37731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:34993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:19.760989 2639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:19.761238 2639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:19.766844 2654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:19.766844 2648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:19.767012 2639 server_base.cc:1034] running on GCE node
W20260809 16:34:19.766878 2650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:19.767398 2639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:19.767724 2639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:19.768920 2639 hybrid_clock.cc:648] HybridClock initialized: now 1786293259768905 us; error 34 us; skew 500 ppm
Aug 09 16:34:19 dist-test-slave-j2vl krb5kdc[2621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293259, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:19.771855 2639 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:34:19.772444 2639 webserver.cc:460] Webserver started at http://127.30.23.190:42357/ using document root <none> and password file <none>
I20260809 16:34:19.772694 2639 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:19.772766 2639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:19.772929 2639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:19.774132 2639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "b95c72b01a514bc8be9302dca9879d02"
format_stamp: "Formatted at 2026-08-09 16:34:19 on dist-test-slave-j2vl"
server_key: "bbcd1410c4728ae475491cfbeed70f5a"
server_key_iv: "915d9b5f7f1dc98d6e0c038f8b03dcf1"
server_key_version: "encryptionkey@0"
I20260809 16:34:19.774530 2639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "b95c72b01a514bc8be9302dca9879d02"
format_stamp: "Formatted at 2026-08-09 16:34:19 on dist-test-slave-j2vl"
server_key: "bbcd1410c4728ae475491cfbeed70f5a"
server_key_iv: "915d9b5f7f1dc98d6e0c038f8b03dcf1"
server_key_version: "encryptionkey@0"
I20260809 16:34:19.776093 2639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:34:19.776971 2663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:19.777237 2639 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:19.777403 2639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "b95c72b01a514bc8be9302dca9879d02"
format_stamp: "Formatted at 2026-08-09 16:34:19 on dist-test-slave-j2vl"
server_key: "bbcd1410c4728ae475491cfbeed70f5a"
server_key_iv: "915d9b5f7f1dc98d6e0c038f8b03dcf1"
server_key_version: "encryptionkey@0"
I20260809 16:34:19.777491 2639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:19.799971 2639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:19.802400 2639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:19.802595 2639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:19.807423 2639 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:34993
I20260809 16:34:19.807461 2745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:34993 every 8 connection(s)
I20260809 16:34:19.807875 2639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:19.809010 2746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:19.810772 2746 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:34:19.815590 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 2639
I20260809 16:34:19.815762 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:34:19.815956 30814 external_mini_cluster.cc:1428] Setting key 91e73e3aee58a0ce5f6336d1c4fd2570
Aug 09 16:34:19 dist-test-slave-j2vl krb5kdc[2621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293259, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:34:19.822075 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:19.811717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43698 (local address 127.30.23.190:34993)
0809 16:34:19.811984 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:19.811989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:19.812053 (+ 64us) server_negotiation.cc:407] Connection header received
0809 16:34:19.813052 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:19.813054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:19.813088 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:19.813174 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:19.813905 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:19.814754 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:19.815674 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:19.816223 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:19.816341 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:19.818634 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:19.818643 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:19.818643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:19.818652 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:19.820346 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:19.820765 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:19.820768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:19.820768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:19.820852 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:19.821087 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:19.821088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:19.821088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:19.821314 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:19.821459 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:19.821678 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 16:34:19.821818 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":81,"threads_started":1}
I20260809 16:34:19.822078 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:19.811567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34993 (local address 127.0.0.1:43698)
0809 16:34:19.811876 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:19.811932 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:19.812742 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:19.813245 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:19.813247 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:19.813603 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:19.813770 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:19.813775 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:19.814864 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:19.814865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:19.815573 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:19.815575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:19.816334 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:19.816336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:19.816396 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:19.816991 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:19.817000 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:19.818525 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:19.820454 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:19.820462 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:19.820465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:19.820666 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:19.820925 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:19.820925 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:19.820926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:19.821018 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:19.821449 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:19.821455 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:19.821523 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 16:34:19.821630 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 16:34:19.821884 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":103,"thread_start_us":157,"threads_started":1}
I20260809 16:34:19.823112 2746 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:34:19.823524 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:19.823379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:38543 (local address 127.0.0.1:43518)
0809 16:34:19.823447 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:19.823493 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 16:34:19.823648 2665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.23.189:38543: connect: Connection refused (error 111)
I20260809 16:34:19.824116 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:19.824019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:38543 (local address 127.0.0.1:43520)
0809 16:34:19.824069 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:19.824093 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 16:34:19.824223 2746 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.189:38543: Network error: Client connection negotiation failed: client connection to 127.30.23.189:38543: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.30.23.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.189.keytab.
Entry for principal kudu/127.30.23.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.189.keytab.
I20260809 16:34:19.850150 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.189:38543
--webserver_interface=127.30.23.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.189.keytab
--principal=kudu/127.30.23.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:34993,127.30.23.189:38543,127.30.23.188:37731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:34:19.873543 2746 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } attempt: 1
I20260809 16:34:19.874090 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:19.873949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:38543 (local address 127.0.0.1:43526)
0809 16:34:19.874019 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:19.874062 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 16:34:19.874532 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:19.874390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:38543 (local address 127.0.0.1:43532)
0809 16:34:19.874456 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:19.874501 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 16:34:19.874665 2746 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.189:38543: Network error: Client connection negotiation failed: client connection to 127.30.23.189:38543: connect: Connection refused (error 111)
W20260809 16:34:19.939849 2755 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:34:19.940085 2755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:19.940104 2755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:19.940126 2755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:19.941366 2755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:19.941418 2755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:19.941433 2755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:19.941447 2755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:19.941459 2755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:19.942854 2746 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } attempt: 2
I20260809 16:34:19.943110 2755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:34993,127.30.23.189:38543,127.30.23.188:37731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.189.keytab
--principal=kudu/127.30.23.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.189:38543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.23.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:19.943416 2755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:19.943429 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:19.943250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:38543 (local address 127.0.0.1:43546)
0809 16:34:19.943359 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:19.943399 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 16:34:19.943687 2755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:34:19.943979 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:19.943844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:38543 (local address 127.0.0.1:43556)
0809 16:34:19.943916 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:19.943952 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:38543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 16:34:19.944084 2746 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.189:38543: Network error: Client connection negotiation failed: client connection to 127.30.23.189:38543: connect: Connection refused (error 111)
W20260809 16:34:19.949429 2765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:19.949422 2766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:19.949458 2755 server_base.cc:1034] running on GCE node
W20260809 16:34:19.949422 2768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:19.949939 2755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:19.950301 2755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:19.951511 2755 hybrid_clock.cc:648] HybridClock initialized: now 1786293259951488 us; error 39 us; skew 500 ppm
Aug 09 16:34:19 dist-test-slave-j2vl krb5kdc[2621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293259, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:19.954463 2755 init.cc:377] Logged in from keytab as kudu/127.30.23.189@KRBTEST.COM (short username kudu)
I20260809 16:34:19.955000 2755 webserver.cc:460] Webserver started at http://127.30.23.189:40847/ using document root <none> and password file <none>
I20260809 16:34:19.955262 2755 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:19.955310 2755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:19.955487 2755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:19.956661 2755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/data/instance:
uuid: "dbf98b5f2c8c4296af3030e073bd858b"
format_stamp: "Formatted at 2026-08-09 16:34:19 on dist-test-slave-j2vl"
server_key: "6ebc5cf951050b300b64cafe4382b7f1"
server_key_iv: "79e41144d1c4bebb86de843739aec17e"
server_key_version: "encryptionkey@0"
I20260809 16:34:19.957041 2755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/wal/instance:
uuid: "dbf98b5f2c8c4296af3030e073bd858b"
format_stamp: "Formatted at 2026-08-09 16:34:19 on dist-test-slave-j2vl"
server_key: "6ebc5cf951050b300b64cafe4382b7f1"
server_key_iv: "79e41144d1c4bebb86de843739aec17e"
server_key_version: "encryptionkey@0"
I20260809 16:34:19.958545 2755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:19.959432 2783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:19.959650 2755 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:19.959751 2755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/wal
uuid: "dbf98b5f2c8c4296af3030e073bd858b"
format_stamp: "Formatted at 2026-08-09 16:34:19 on dist-test-slave-j2vl"
server_key: "6ebc5cf951050b300b64cafe4382b7f1"
server_key_iv: "79e41144d1c4bebb86de843739aec17e"
server_key_version: "encryptionkey@0"
I20260809 16:34:19.959842 2755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:19.974467 2755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:19.977039 2755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:19.977216 2755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:19.981495 2755 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.189:38543
I20260809 16:34:19.981549 2861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.189:38543 every 8 connection(s)
I20260809 16:34:19.982034 2755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/data/info.pb
I20260809 16:34:19.983213 2863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:19.985045 2863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:34:19.986366 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 2755
I20260809 16:34:19.986485 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-1/wal/instance
I20260809 16:34:19.986668 30814 external_mini_cluster.cc:1428] Setting key 449676d37b2f211a214ee0d469a89ddb
Aug 09 16:34:19 dist-test-slave-j2vl krb5kdc[2621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293259, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.189@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.23.188@KRBTEST.COM; defaulting to no policy
I20260809 16:34:19.996562 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:19.985989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43700 (local address 127.30.23.190:34993)
0809 16:34:19.986059 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:19.986061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:19.986312 (+ 251us) server_negotiation.cc:407] Connection header received
0809 16:34:19.986931 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:19.986933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:19.986947 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:19.987029 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:19.987674 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:19.988444 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:19.990074 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:19.990290 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:19.990359 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:19.993053 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:19.993055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:19.993055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:19.993065 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:19.994767 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:19.995288 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:19.995290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:19.995290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:19.995363 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:19.995715 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:19.995716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:19.995717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:19.995845 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:19.995936 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:19.996266 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 16:34:19.996366 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:34:19.996775 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:19.985903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34993 (local address 127.0.0.1:43700)
0809 16:34:19.986163 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:19.986208 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:19.986806 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:19.987099 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:19.987102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:19.987364 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:19.987580 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:19.987584 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:19.988563 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:19.988564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:19.989962 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:19.989966 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:19.990445 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:19.990447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:19.990615 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:19.991194 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:19.991206 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:19.992943 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:19.994915 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:19.994924 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:19.994927 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:19.995196 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:19.995450 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:19.995451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:19.995452 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:19.995581 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:19.995938 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:19.995947 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:19.996110 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 16:34:19.996229 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 16:34:19.996514 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":137,"threads_started":1}
I20260809 16:34:19.997699 2863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.188@KRBTEST.COM" created.
Aug 09 16:34:19 dist-test-slave-j2vl krb5kdc[2621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293259, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.189@KRBTEST.COM for kudu/127.30.23.189@KRBTEST.COM
I20260809 16:34:20.007529 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:19.998000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:38543 (local address 127.0.0.1:43570)
0809 16:34:19.998058 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:19.998071 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:19.998120 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:19.998472 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:19.998473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:19.998610 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:19.998670 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:19.998671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:19.999775 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:19.999777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.000394 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:20.000397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.001335 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.001336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.001445 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.001953 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:20.001959 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:20.003530 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:20.006118 (+ 2588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.006119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.006119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.006385 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.006641 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.006642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.006642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.006720 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.007084 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:20.007085 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:20.007143 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 16:34:20.007226 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 16:34:20.007351 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 16:34:20.007531 2869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:19.998110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43570 (local address 127.30.23.189:38543)
0809 16:34:19.998288 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:19.998292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:19.998317 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:34:19.998340 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:19.998341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:19.998352 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:19.998423 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:19.998761 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:19.999657 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.000508 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.001095 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.001193 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.003628 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:20.003640 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:20.003640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:20.003649 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:20.005982 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.006492 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.006496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.006496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.006559 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.006827 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.006827 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.006828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.007007 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:20.007077 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:20.007264 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 16:34:20.007424 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260809 16:34:20.008414 2863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:34:20.008816 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.008666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38182)
0809 16:34:20.008721 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.008784 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 16:34:20.008976 2785 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
I20260809 16:34:20.009533 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.009432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38186)
0809 16:34:20.009488 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.009514 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 16:34:20.009672 2863 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:37731: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.188.keytab.
Entry for principal kudu/127.30.23.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.188.keytab.
I20260809 16:34:20.023123 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.188:37731
--webserver_interface=127.30.23.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.188.keytab
--principal=kudu/127.30.23.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:34993,127.30.23.189:38543,127.30.23.188:37731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:34:20.035254 2746 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } attempt: 3
Aug 09 16:34:20 dist-test-slave-j2vl krb5kdc[2621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293259, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for kudu/127.30.23.189@KRBTEST.COM
I20260809 16:34:20.042927 2869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.035772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43576 (local address 127.30.23.189:38543)
0809 16:34:20.035855 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.035857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.035869 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:34:20.035957 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.035958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.035969 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.036035 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.036421 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.036915 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.037703 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.037936 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.038029 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.040135 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:20.040137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:20.040137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:20.040144 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:20.041781 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.042203 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.042204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.042204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.042259 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.042460 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.042461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.042461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.042569 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:20.042630 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:20.042791 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 16:34:20.042858 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 16:34:20.042977 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.035686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:38543 (local address 127.0.0.1:43576)
0809 16:34:20.035754 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.035768 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:20.035846 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:20.036042 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:20.036043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:20.036228 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:20.036298 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:20.036300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.037002 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.037004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.037612 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:20.037614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.038029 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.038030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.038112 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.038610 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:20.038614 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:20.040009 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:20.041875 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.041876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.041877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.042118 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.042328 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.042329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.042329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.042399 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.042640 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:20.042641 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:20.042695 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 16:34:20.042765 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 16:34:20.042870 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 16:34:20.043689 2746 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:34:20.044117 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.043999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38196)
0809 16:34:20.044051 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.044089 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 16:34:20.044509 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.044414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38208)
0809 16:34:20.044462 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.044490 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 16:34:20.044665 2746 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:37731: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
I20260809 16:34:20.058980 2863 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } attempt: 1
I20260809 16:34:20.059578 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.059422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38218)
0809 16:34:20.059497 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.059544 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 16:34:20.060148 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.059973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38222)
0809 16:34:20.060088 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.060124 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260809 16:34:20.060295 2863 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:37731: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
I20260809 16:34:20.075839 2746 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } attempt: 1
I20260809 16:34:20.076421 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.076267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38228)
0809 16:34:20.076335 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.076386 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 16:34:20.076812 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.076714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38236)
0809 16:34:20.076764 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.076792 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 16:34:20.076915 2746 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:37731: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
W20260809 16:34:20.118266 2873 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:34:20.118515 2873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:20.118533 2873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:20.118556 2873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:20.119866 2873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:20.119923 2873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:20.119938 2873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:20.119952 2873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:20.119964 2873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:20.122007 2873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:34993,127.30.23.189:38543,127.30.23.188:37731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.188.keytab
--principal=kudu/127.30.23.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.188:37731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.23.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:20.122336 2873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:20.122639 2873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:34:20.128448 2863 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } attempt: 2
W20260809 16:34:20.128438 2879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:20.128479 2883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:20.128834 2880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:20.128883 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.128734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38252)
0809 16:34:20.128799 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.128844 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 16:34:20.128873 2873 server_base.cc:1034] running on GCE node
I20260809 16:34:20.129194 2873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:20.129369 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.129198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38256)
0809 16:34:20.129255 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.129311 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 16:34:20.129460 2863 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:37731: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
I20260809 16:34:20.129621 2873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:20.130796 2873 hybrid_clock.cc:648] HybridClock initialized: now 1786293260130779 us; error 28 us; skew 500 ppm
Aug 09 16:34:20 dist-test-slave-j2vl krb5kdc[2621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293260, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:20.133673 2873 init.cc:377] Logged in from keytab as kudu/127.30.23.188@KRBTEST.COM (short username kudu)
I20260809 16:34:20.134195 2873 webserver.cc:460] Webserver started at http://127.30.23.188:40999/ using document root <none> and password file <none>
I20260809 16:34:20.134439 2873 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:20.134510 2873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:20.134651 2873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:20.135824 2873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/data/instance:
uuid: "ae0b9981299e4a8bb39823a758d173a4"
format_stamp: "Formatted at 2026-08-09 16:34:20 on dist-test-slave-j2vl"
server_key: "81654e1611869756d8a12b460514a7e9"
server_key_iv: "6d5b6727327f4717ece951eecc07d560"
server_key_version: "encryptionkey@0"
I20260809 16:34:20.136214 2873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/wal/instance:
uuid: "ae0b9981299e4a8bb39823a758d173a4"
format_stamp: "Formatted at 2026-08-09 16:34:20 on dist-test-slave-j2vl"
server_key: "81654e1611869756d8a12b460514a7e9"
server_key_iv: "6d5b6727327f4717ece951eecc07d560"
server_key_version: "encryptionkey@0"
I20260809 16:34:20.137669 2873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:20.138550 2892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:20.138870 2873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:20.138988 2873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/wal
uuid: "ae0b9981299e4a8bb39823a758d173a4"
format_stamp: "Formatted at 2026-08-09 16:34:20 on dist-test-slave-j2vl"
server_key: "81654e1611869756d8a12b460514a7e9"
server_key_iv: "6d5b6727327f4717ece951eecc07d560"
server_key_version: "encryptionkey@0"
I20260809 16:34:20.139068 2873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:20.152060 2746 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } attempt: 2
I20260809 16:34:20.152611 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.152466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38258)
0809 16:34:20.152530 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.152577 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 16:34:20.153071 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.152914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38274)
0809 16:34:20.153010 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.153044 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 16:34:20.153209 2746 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:37731: Network error: Client connection negotiation failed: client connection to 127.30.23.188:37731: connect: Connection refused (error 111)
I20260809 16:34:20.169889 2873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:20.172395 2873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:20.172592 2873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:20.177138 2873 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.188:37731
I20260809 16:34:20.177201 2976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.188:37731 every 8 connection(s)
I20260809 16:34:20.177613 2873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/data/info.pb
I20260809 16:34:20.178659 2981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:20.180032 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 2873
I20260809 16:34:20.180112 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786292963539133-30814-0/minicluster-data/master-2/wal/instance
I20260809 16:34:20.180344 30814 external_mini_cluster.cc:1428] Setting key ab4f643c3bacbd7cf28b016c2f3e8dc3
I20260809 16:34:20.180501 2981 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:34:20 dist-test-slave-j2vl krb5kdc[2621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293260, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.188@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
Aug 09 16:34:20 dist-test-slave-j2vl krb5kdc[2621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.188@KRBTEST.COM
I20260809 16:34:20.191574 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.181213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43716 (local address 127.30.23.190:34993)
0809 16:34:20.181280 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.181282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.181500 (+ 218us) server_negotiation.cc:407] Connection header received
0809 16:34:20.181840 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.181841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.181854 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.181932 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.182577 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.183064 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.184276 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.184551 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.184626 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.187629 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:20.187630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:20.187630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:20.187638 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:20.189708 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.190285 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.190286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.190287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.190510 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.190826 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.190827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.190827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.191003 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:20.191083 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:20.191372 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 16:34:20.191469 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:34:20.191821 2987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.181187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:34993 (local address 127.0.0.1:43716)
0809 16:34:20.181370 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.181409 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:20.181738 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:20.181954 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:20.181959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:20.182268 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:20.182470 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:20.182480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.183148 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.183149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.184181 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:20.184184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.184659 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.184660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.184763 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.185673 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:20.185684 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:20.187512 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:20.189797 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.189802 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.189805 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.190182 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.190597 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.190598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.190598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.190740 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.191078 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:20.191085 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:20.191209 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 16:34:20.191331 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 16:34:20.191531 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260809 16:34:20.191823 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.180865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38282 (local address 127.30.23.188:37731)
0809 16:34:20.181048 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.181058 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.181134 (+ 76us) server_negotiation.cc:407] Connection header received
0809 16:34:20.181727 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.181734 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.181780 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.182563 (+ 783us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.182606 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.183364 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.184418 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.184834 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.184889 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.188134 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:20.188144 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:20.188144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:20.188151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:20.190279 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.190661 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.190664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.190664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.190763 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.190994 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.190995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.190995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.191216 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:20.191363 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:20.191450 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 16:34:20.191601 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260809 16:34:20.192647 2981 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:34:20 dist-test-slave-j2vl krb5kdc[2621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293260, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.188@KRBTEST.COM for kudu/127.30.23.189@KRBTEST.COM
Aug 09 16:34:20 dist-test-slave-j2vl krb5kdc[2621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:34:20.200344 2987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.192938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:38543 (local address 127.0.0.1:43592)
0809 16:34:20.192982 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.192991 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:20.193022 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:20.193196 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:20.193197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:20.193501 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:20.193559 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:20.193561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.194230 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.194232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.194746 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:20.194748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.195121 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.195123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.195191 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.195702 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:20.195706 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:20.197163 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:20.199119 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.199120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.199121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.199326 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.199548 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.199549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.199549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.199625 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.200023 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:20.200025 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:20.200079 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 16:34:20.200157 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 16:34:20.200265 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 16:34:20.200418 2869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.193007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43592 (local address 127.30.23.189:38543)
0809 16:34:20.193057 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.193059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.193068 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:34:20.193093 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.193093 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.193110 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.193175 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.193653 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.194145 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.194833 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.195048 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.195127 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.197256 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:20.197257 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:20.197258 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:20.197267 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:20.198978 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.199427 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.199428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.199428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.199480 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.199729 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.199756 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.199756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.199937 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:20.200016 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:20.200223 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 16:34:20.200311 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:34:20.200932 2981 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:34:20.201231 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.193013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43724 (local address 127.30.23.190:34993)
0809 16:34:20.193057 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.193060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.193067 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:34:20.193087 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.193088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.193098 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.193150 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.193720 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.194183 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.194882 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.195079 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.195155 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.197820 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:20.197821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:20.197822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:20.197829 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:20.199299 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.199789 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.199790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.199791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.199850 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.200175 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.200176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.200176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.200321 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:20.200512 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:20.201010 (+ 498us) server_negotiation.cc:299] Negotiation successful
0809 16:34:20.201099 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 16:34:20 dist-test-slave-j2vl krb5kdc[2621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293260, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.188@KRBTEST.COM for kudu/127.30.23.188@KRBTEST.COM
Aug 09 16:34:20 dist-test-slave-j2vl krb5kdc[2621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.189@KRBTEST.COM
I20260809 16:34:20.207823 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.201300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38288 (local address 127.30.23.188:37731)
0809 16:34:20.201400 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.201402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.201413 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:34:20.201437 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.201437 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.201448 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.201518 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.201776 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.202406 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.203088 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.203300 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.203389 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.205134 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:20.205136 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:20.205136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:20.205144 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:20.206659 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.207012 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.207013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.207013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.207069 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.207256 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.207257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.207258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.207381 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:20.207452 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:20.207600 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 16:34:20.207693 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":79}
I20260809 16:34:20.207839 2987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.201250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38288)
0809 16:34:20.201307 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.201318 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:20.201388 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:20.201524 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:20.201524 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:20.201659 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:20.201691 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:20.201692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.202507 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.202508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.202996 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:20.202997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.203388 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.203389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.203437 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.203893 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:20.203899 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:20.205050 (+ 1151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:20.206744 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.206745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.206745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.206919 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.207133 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.207134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.207134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.207193 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.207455 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:20.207457 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:20.207494 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 16:34:20.207578 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 16:34:20.207742 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":72}
I20260809 16:34:20.209322 2981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4: Bootstrap starting.
I20260809 16:34:20.209440 2869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.202220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43598 (local address 127.30.23.189:38543)
0809 16:34:20.202281 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.202283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.202290 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:34:20.202310 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.202311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.202320 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.202391 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.202902 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.203336 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.203992 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.204210 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.204284 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.206680 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:20.206682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:20.206682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:20.206690 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:20.208252 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.208602 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.208604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.208604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.208667 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.208870 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.208872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.208872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.208988 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:20.209178 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:20.209216 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 16:34:20.209291 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:34:20.210285 2981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:20.210623 2981 log.cc:824] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:20.211005 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:20.211495 2981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4: No bootstrap required, opened a new log
I20260809 16:34:20.212016 2704 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b95c72b01a514bc8be9302dca9879d02"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43724
I20260809 16:34:20.213804 2981 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } }
I20260809 16:34:20.213980 2981 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0b9981299e4a8bb39823a758d173a4, State: Initialized, Role: FOLLOWER
I20260809 16:34:20.214061 2981 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } }
I20260809 16:34:20.214407 2981 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:20.214378 2989 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } } }
I20260809 16:34:20.214496 2989 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:34:20.216481 3011 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:34:20.216547 3011 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:34:20.216863 2873 master_runner.cc:186] Error getting master registration for 127.30.23.190:34993: OK, instance_id { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" instance_seqno: 1786293259806070 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:34:20.219808 2873 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:34:20.220613 2863 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } attempt: 3
Aug 09 16:34:20 dist-test-slave-j2vl krb5kdc[2621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293259, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.189@KRBTEST.COM for kudu/127.30.23.188@KRBTEST.COM
I20260809 16:34:20.229758 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.221175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38298)
0809 16:34:20.221307 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.221318 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:20.221434 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:20.221620 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:20.221621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:20.221869 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:20.221929 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:20.221932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.222855 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.222856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.223728 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:20.223730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.224277 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.224278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.224353 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.224936 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:20.224942 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:20.226504 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:20.228418 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.228419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.228420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.228661 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.228940 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.228941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.228942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.229033 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.229388 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:20.229389 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:20.229443 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 16:34:20.229525 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:34:20.229637 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260809 16:34:20.229931 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.221249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.30.23.188:37731)
0809 16:34:20.221299 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.221302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.221393 (+ 91us) server_negotiation.cc:407] Connection header received
0809 16:34:20.221507 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.221508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.221519 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.221595 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.222017 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.222760 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.223834 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.224146 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.224227 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.226608 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:20.226610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:20.226610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:20.226618 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:20.228319 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.228789 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.228790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.228790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.228859 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.229087 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.229088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.229088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.229289 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:20.229390 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:20.229617 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 16:34:20.229725 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:34:20.232121 2863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b: Bootstrap starting.
I20260809 16:34:20.232918 2863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:20.233229 2863 log.cc:824] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:20.234098 2863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b: No bootstrap required, opened a new log
I20260809 16:34:20.235759 2863 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } }
I20260809 16:34:20.235972 2863 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbf98b5f2c8c4296af3030e073bd858b, State: Initialized, Role: FOLLOWER
I20260809 16:34:20.236157 2863 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } }
I20260809 16:34:20.236513 3015 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } } }
I20260809 16:34:20.236613 2863 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:20.236619 3015 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:34:20.239434 3032 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:34:20.239535 3032 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:34:20.239434 2755 master_runner.cc:186] Error getting master registration for 127.30.23.190:34993: OK, instance_id { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" instance_seqno: 1786293259806070 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:34:20.242431 2755 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:34:20.252372 2746 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } attempt: 3
Aug 09 16:34:20 dist-test-slave-j2vl krb5kdc[2621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293259, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for kudu/127.30.23.188@KRBTEST.COM
I20260809 16:34:20.260753 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.252873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38304 (local address 127.30.23.188:37731)
0809 16:34:20.252919 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.252921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.252933 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:34:20.253012 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.253013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.253039 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.253093 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.253517 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.254363 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.255104 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.255413 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.255506 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.257676 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:20.257678 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:20.257678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:20.257729 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:20.259550 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.259908 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.259910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.259910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.259974 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.260182 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.260183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.260184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.260336 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:20.260401 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:20.260567 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 16:34:20.260651 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:34:20.260753 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.252810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:37731 (local address 127.0.0.1:38304)
0809 16:34:20.252886 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:20.252899 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:20.252952 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:20.253107 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:20.253108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:20.253305 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:20.253425 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:20.253428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.254484 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.254485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.255010 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:20.255012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.255515 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.255517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:20.255581 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.256084 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:20.256089 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:20.257588 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:20.259633 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.259634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.259635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.259826 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.260051 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:20.260051 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:20.260051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:20.260114 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:20.260399 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:20.260400 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:20.260451 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 16:34:20.260541 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 16:34:20.260651 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 16:34:20.263195 2746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02: Bootstrap starting.
I20260809 16:34:20.264000 2746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:20.264299 2746 log.cc:824] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:20.265209 2746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02: No bootstrap required, opened a new log
I20260809 16:34:20.266677 2746 raft_consensus.cc:348] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } }
I20260809 16:34:20.266841 2746 raft_consensus.cc:729] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b95c72b01a514bc8be9302dca9879d02, State: Initialized, Role: FOLLOWER
I20260809 16:34:20.266932 2746 consensus_queue.cc:260] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } }
I20260809 16:34:20.267300 2746 sys_catalog.cc:564] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:20.267330 2704 raft_consensus.cc:482] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 16:34:20.267376 2704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:20.268317 2704 raft_consensus.cc:504] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } }
I20260809 16:34:20.268607 2704 leader_election.cc:288] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [CANDIDATE]: Term 1 election: Requested vote from peers dbf98b5f2c8c4296af3030e073bd858b (127.30.23.189:38543), ae0b9981299e4a8bb39823a758d173a4 (127.30.23.188:37731)
I20260809 16:34:20.269299 2817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b95c72b01a514bc8be9302dca9879d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dbf98b5f2c8c4296af3030e073bd858b"
I20260809 16:34:20.269495 2817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:20.270311 2935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b95c72b01a514bc8be9302dca9879d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ae0b9981299e4a8bb39823a758d173a4"
I20260809 16:34:20.270452 2935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:20.270535 2817 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b95c72b01a514bc8be9302dca9879d02 in term 1.
I20260809 16:34:20.271286 2665 leader_election.cc:302] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b95c72b01a514bc8be9302dca9879d02, dbf98b5f2c8c4296af3030e073bd858b; no voters:
I20260809 16:34:20.271415 2935 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b95c72b01a514bc8be9302dca9879d02 in term 1.
I20260809 16:34:20.267262 3035 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } } }
I20260809 16:34:20.271917 3035 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:34:20.272017 3035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:20.272107 3035 raft_consensus.cc:686] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [term 1 LEADER]: Becoming Leader. State: Replica: b95c72b01a514bc8be9302dca9879d02, State: Running, Role: LEADER
I20260809 16:34:20.272240 3035 consensus_queue.cc:237] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } }
I20260809 16:34:20.272958 2639 master_runner.cc:263] No leader master found from master b95c72b01a514bc8be9302dca9879d02
I20260809 16:34:20.273011 2639 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:34:20.273312 3035 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b95c72b01a514bc8be9302dca9879d02. Latest consensus state: current_term: 1 leader_uuid: "b95c72b01a514bc8be9302dca9879d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } } }
I20260809 16:34:20.273485 3035 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:20.273764 3061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:20.274340 3061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:20.275852 2935 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4 [term 1 FOLLOWER]: Refusing update from remote peer b95c72b01a514bc8be9302dca9879d02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:20.276275 2817 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b [term 1 FOLLOWER]: Refusing update from remote peer b95c72b01a514bc8be9302dca9879d02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:20.276510 3035 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:20.277920 2989 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b95c72b01a514bc8be9302dca9879d02. Latest consensus state: current_term: 1 leader_uuid: "b95c72b01a514bc8be9302dca9879d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } } }
I20260809 16:34:20.278008 2989 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:34:20.278136 3068 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 16:34:20.279439 3015 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b95c72b01a514bc8be9302dca9879d02. Latest consensus state: current_term: 1 leader_uuid: "b95c72b01a514bc8be9302dca9879d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } } }
I20260809 16:34:20.279532 3015 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:34:20.284605 2986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.270259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38314 (local address 127.30.23.188:37731)
0809 16:34:20.270301 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.270304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.275137 (+ 4833us) server_negotiation.cc:407] Connection header received
0809 16:34:20.275244 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.275245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.275256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.275287 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.277498 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.278211 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.279243 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.279518 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.279568 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.281466 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:20.281468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:20.281468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:20.281476 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:20.283218 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.283669 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.283670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.283671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.283733 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.283950 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.283951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.283951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.284095 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:20.284179 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:20.284415 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 16:34:20.284505 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:34:20.291074 3058 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b95c72b01a514bc8be9302dca9879d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } } }
I20260809 16:34:20.291196 3058 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:20.291289 3064 mvcc.cc:205] Tried to move back new op lower bound from 7316657194087161856 to 7316657194078322688. Current Snapshot: MvccSnapshot[applied={T|T < 7316657194087161856}]
I20260809 16:34:20.291638 3035 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b95c72b01a514bc8be9302dca9879d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } } }
I20260809 16:34:20.291733 3035 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:20.292277 3061 catalog_manager.cc:1347] Generated new cluster ID: f64329dd6c4c4c48b5cbdf65614b312d
I20260809 16:34:20.292322 3061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:20.293614 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.270653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43726 (local address 127.30.23.190:34993)
0809 16:34:20.271563 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.271566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.285477 (+ 13911us) server_negotiation.cc:407] Connection header received
0809 16:34:20.285580 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.285581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.285594 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.285627 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.286275 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.287022 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.288039 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.288351 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.288397 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.290036 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:20.290038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:20.290038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:20.290047 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:20.292286 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.292807 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.292808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.292809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.292862 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.293041 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.293041 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.293041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.293155 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:20.293218 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:20.293450 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 16:34:20.293530 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":888}
I20260809 16:34:20.294132 2869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:20.277136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43612 (local address 127.30.23.189:38543)
0809 16:34:20.277181 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.277184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.285516 (+ 8332us) server_negotiation.cc:407] Connection header received
0809 16:34:20.285635 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.285637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.285649 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.285682 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.286467 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.287288 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.288390 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:20.288714 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:20.288758 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:20.290819 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:20.290821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:20.290822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:20.290830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:20.292709 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.293152 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.293153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.293154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.293215 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:20.293483 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:20.293484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:20.293485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:20.293639 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:20.293714 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:20.293940 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 16:34:20.294031 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:34:20.297731 3015 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b95c72b01a514bc8be9302dca9879d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } } }
I20260809 16:34:20.297847 3015 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:34:20.298606 2989 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b95c72b01a514bc8be9302dca9879d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b95c72b01a514bc8be9302dca9879d02" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 34993 } } peers { permanent_uuid: "dbf98b5f2c8c4296af3030e073bd858b" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 38543 } } peers { permanent_uuid: "ae0b9981299e4a8bb39823a758d173a4" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 37731 } } }
I20260809 16:34:20.298718 2989 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:34:20.299182 3076 mvcc.cc:205] Tried to move back new op lower bound from 7316657194087161856 to 7316657194078322688. Current Snapshot: MvccSnapshot[applied={T|T < 7316657194087161856}]
I20260809 16:34:20.313663 3061 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:20.314682 3061 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:20.328804 3061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b95c72b01a514bc8be9302dca9879d02: Generated new TSK 0
I20260809 16:34:20.329090 3061 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:34:20.356875 2986 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:20.354967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38318 (local address 127.30.23.188:37731)
0809 16:34:20.355017 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.355020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.355940 (+ 920us) server_negotiation.cc:407] Connection header received
0809 16:34:20.356167 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.356168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.356183 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.356220 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.356795 (+ 575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38318: BlockingRecv error: recv got EOF from 127.0.0.1:38318 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 16:34:20.357679 2869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:20.354678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43616 (local address 127.30.23.189:38543)
0809 16:34:20.354762 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.354765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.356105 (+ 1340us) server_negotiation.cc:407] Connection header received
0809 16:34:20.357130 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.357132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.357145 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.357179 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.357615 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43616: BlockingRecv error: recv got EOF from 127.0.0.1:43616 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 16:34:20.358582 2749 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:20.355516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43734 (local address 127.30.23.190:34993)
0809 16:34:20.355563 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.355566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.358000 (+ 2434us) server_negotiation.cc:407] Connection header received
0809 16:34:20.358124 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.358126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.358138 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.358171 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.358515 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43734: BlockingRecv error: recv got EOF from 127.0.0.1:43734 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 16:34:20.361310 2869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:20.360407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43620 (local address 127.30.23.189:38543)
0809 16:34:20.360448 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.360450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.360711 (+ 261us) server_negotiation.cc:407] Connection header received
0809 16:34:20.360846 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.360848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.360869 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.360902 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.361255 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43620: BlockingRecv error: recv got EOF from 127.0.0.1:43620 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 16:34:20.364663 2869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:20.363852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43628 (local address 127.30.23.189:38543)
0809 16:34:20.363888 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.363890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.364121 (+ 231us) server_negotiation.cc:407] Connection header received
0809 16:34:20.364226 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.364227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.364239 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.364272 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.364612 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43628: BlockingRecv error: recv got EOF from 127.0.0.1:43628 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 16:34:20.368875 2869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:20.368028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43634 (local address 127.30.23.189:38543)
0809 16:34:20.368068 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.368070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.368309 (+ 239us) server_negotiation.cc:407] Connection header received
0809 16:34:20.368410 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.368411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.368423 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.368453 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.368821 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43634: BlockingRecv error: recv got EOF from 127.0.0.1:43634 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 16:34:20.375169 2869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:20.374277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43644 (local address 127.30.23.189:38543)
0809 16:34:20.374320 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.374322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.374555 (+ 233us) server_negotiation.cc:407] Connection header received
0809 16:34:20.374658 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.374659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.374670 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.374723 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.375100 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43644: BlockingRecv error: recv got EOF from 127.0.0.1:43644 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:34:20.385670 2869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:20.384619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43658 (local address 127.30.23.189:38543)
0809 16:34:20.384667 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.384670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.384932 (+ 262us) server_negotiation.cc:407] Connection header received
0809 16:34:20.385033 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.385034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.385046 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.385076 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.385605 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43658: BlockingRecv error: recv got EOF from 127.0.0.1:43658 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:34:20.404250 2869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:20.403219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43660 (local address 127.30.23.189:38543)
0809 16:34:20.403273 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.403276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.403559 (+ 283us) server_negotiation.cc:407] Connection header received
0809 16:34:20.403676 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.403677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.403691 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.403739 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.404186 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43660: BlockingRecv error: recv got EOF from 127.0.0.1:43660 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 16:34:20.438778 2869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:20.437921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43662 (local address 127.30.23.189:38543)
0809 16:34:20.437983 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.437985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.438151 (+ 166us) server_negotiation.cc:407] Connection header received
0809 16:34:20.438224 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.438225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.438245 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.438308 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.438703 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43662: BlockingRecv error: recv got EOF from 127.0.0.1:43662 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260809 16:34:20.505298 2869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:20.504661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43674 (local address 127.30.23.189:38543)
0809 16:34:20.504739 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.504741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.504751 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:34:20.504778 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.504778 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.504789 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.504847 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.505230 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43674: BlockingRecv error: recv got EOF from 127.0.0.1:43674 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 16:34:20.635576 2869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:20.634656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43690 (local address 127.30.23.189:38543)
0809 16:34:20.634741 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.634743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.634799 (+ 56us) server_negotiation.cc:407] Connection header received
0809 16:34:20.634852 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.634853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.634863 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.634916 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.635526 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43690: BlockingRecv error: recv got EOF from 127.0.0.1:43690 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 16:34:20.893910 2869 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:20.893144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.30.23.189:38543)
0809 16:34:20.893220 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:20.893222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:20.893233 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:34:20.893289 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:20.893290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:20.893303 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:20.893423 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:20.893820 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43694: BlockingRecv error: recv got EOF from 127.0.0.1:43694 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:34:21.217044 3011 catalog_manager.cc:1259] Loaded cluster ID: f64329dd6c4c4c48b5cbdf65614b312d
I20260809 16:34:21.217165 3011 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4: loading cluster ID for follower catalog manager: success
I20260809 16:34:21.218010 3011 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4: acquiring CA information for follower catalog manager: success
I20260809 16:34:21.218325 3011 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ae0b9981299e4a8bb39823a758d173a4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:34:21.240052 3032 catalog_manager.cc:1259] Loaded cluster ID: f64329dd6c4c4c48b5cbdf65614b312d
I20260809 16:34:21.240161 3032 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b: loading cluster ID for follower catalog manager: success
I20260809 16:34:21.241000 3032 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b: acquiring CA information for follower catalog manager: success
I20260809 16:34:21.241391 3032 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dbf98b5f2c8c4296af3030e073bd858b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:34:21.410422 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:21.407349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43704 (local address 127.30.23.189:38543)
0809 16:34:21.407495 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:21.407497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:21.407512 (+ 15us) server_negotiation.cc:407] Connection header received
0809 16:34:21.407626 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:21.407627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:21.407696 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:21.407829 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:21.408005 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:21.408566 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:21.409454 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:21.409689 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:21.409792 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:21.410011 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:21.410135 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:21.410236 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:21.410279 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 16:34:21.410305 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260809 16:34:21.415124 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:21.412488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38334 (local address 127.30.23.188:37731)
0809 16:34:21.412615 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:21.412617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:21.412628 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:34:21.412681 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:21.412682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:21.412721 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:21.412804 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:21.412892 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:21.413439 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:21.414162 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:21.414502 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:21.414575 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:21.414761 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:21.414873 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:21.414948 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:21.415031 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 16:34:21.415051 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":82,"threads_started":1}
I20260809 16:34:21.419934 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:21.416974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43714 (local address 127.30.23.189:38543)
0809 16:34:21.417037 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:21.417039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:21.417216 (+ 177us) server_negotiation.cc:407] Connection header received
0809 16:34:21.417326 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:21.417327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:21.417361 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:21.417425 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:21.417559 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:21.418154 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:21.419100 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:21.419392 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:21.419444 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:21.419652 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:21.419688 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:21.419719 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:21.419820 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 16:34:21.419841 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:34:21.420104 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:21.417074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38348 (local address 127.30.23.188:37731)
0809 16:34:21.417122 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:21.417124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:21.417527 (+ 403us) server_negotiation.cc:407] Connection header received
0809 16:34:21.417622 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:21.417623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:21.417635 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:21.417663 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:21.417845 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:21.418529 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:21.419278 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:21.419524 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:21.419587 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:21.419730 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:21.419768 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:21.419821 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:21.420009 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 16:34:21.420036 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:34:21.420572 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:21.416973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43744 (local address 127.30.23.190:34993)
0809 16:34:21.417175 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:21.417177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:21.417283 (+ 106us) server_negotiation.cc:407] Connection header received
0809 16:34:21.417389 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:21.417390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:21.417444 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:21.417537 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:21.417666 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:21.418432 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:21.419468 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:21.419783 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:21.419843 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:21.420086 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:21.420193 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:21.420260 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:21.420467 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 16:34:21.420493 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":54,"threads_started":1}
I20260809 16:34:21.422583 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 2639
I20260809 16:34:21.436580 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 2755
I20260809 16:34:21.451195 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 2873
2026-08-09T16:34:21Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3250 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:21 dist-test-slave-j2vl krb5kdc[3213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:21 dist-test-slave-j2vl krb5kdc[3213](info): set up 2 sockets
Aug 09 16:34:21 dist-test-slave-j2vl krb5kdc[3213](info): commencing operation
krb5kdc: starting...
W20260809 16:34:22.788265 30814 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/test-user.keytab.
Aug 09 16:34:22 dist-test-slave-j2vl krb5kdc[3213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.23.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
Entry for principal kudu/127.30.23.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab.
I20260809 16:34:22.929551 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:39343
--webserver_interface=127.30.23.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:39343,127.30.23.189:43603,127.30.23.188:42445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:23.015475 3232 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:34:23.015700 3232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:23.015722 3232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:23.015744 3232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:23.016958 3232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:23.017010 3232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:23.017024 3232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:23.017037 3232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:23.017050 3232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:23.018666 3232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:39343,127.30.23.189:43603,127.30.23.188:42445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.190.keytab
--principal=kudu/127.30.23.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.190:39343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.23.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:23.018937 3232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:23.019213 3232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:23.024638 3238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:23.024667 3232 server_base.cc:1034] running on GCE node
W20260809 16:34:23.024633 3239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:23.024629 3242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:23.025094 3232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:23.025439 3232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:23.026623 3232 hybrid_clock.cc:648] HybridClock initialized: now 1786293263026607 us; error 32 us; skew 500 ppm
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:23.034330 3232 init.cc:377] Logged in from keytab as kudu/127.30.23.190@KRBTEST.COM (short username kudu)
I20260809 16:34:23.034999 3232 webserver.cc:460] Webserver started at http://127.30.23.190:34589/ using document root <none> and password file <none>
I20260809 16:34:23.035256 3232 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:23.035307 3232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:23.035462 3232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:23.036595 3232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/data/instance:
uuid: "02cfcb8b64384d9f9476907c6813378f"
format_stamp: "Formatted at 2026-08-09 16:34:23 on dist-test-slave-j2vl"
server_key: "70fde4bcb347001cfd814f3cd0e1464b"
server_key_iv: "fe8e0d9b27d892f9ff033b0adcc52855"
server_key_version: "encryptionkey@0"
I20260809 16:34:23.036983 3232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/wal/instance:
uuid: "02cfcb8b64384d9f9476907c6813378f"
format_stamp: "Formatted at 2026-08-09 16:34:23 on dist-test-slave-j2vl"
server_key: "70fde4bcb347001cfd814f3cd0e1464b"
server_key_iv: "fe8e0d9b27d892f9ff033b0adcc52855"
server_key_version: "encryptionkey@0"
I20260809 16:34:23.038480 3232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:34:23.039350 3251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:23.039673 3232 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:34:23.039852 3232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/wal
uuid: "02cfcb8b64384d9f9476907c6813378f"
format_stamp: "Formatted at 2026-08-09 16:34:23 on dist-test-slave-j2vl"
server_key: "70fde4bcb347001cfd814f3cd0e1464b"
server_key_iv: "fe8e0d9b27d892f9ff033b0adcc52855"
server_key_version: "encryptionkey@0"
I20260809 16:34:23.039980 3232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:23.051054 3232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:23.053313 3232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:23.053535 3232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:23.058604 3232 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.190:39343
I20260809 16:34:23.058893 3337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.190:39343 every 8 connection(s)
I20260809 16:34:23.059113 3232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:23.060458 3338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:23.062418 3338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:34:23.066120 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 3232
I20260809 16:34:23.066460 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-0/wal/instance
I20260809 16:34:23.066689 30814 external_mini_cluster.cc:1428] Setting key 5ad7ce96996d2a36d7ab6516facb6c61
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.23.189@KRBTEST.COM; defaulting to no policy
I20260809 16:34:23.075685 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.063470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42932 (local address 127.30.23.190:39343)
0809 16:34:23.063650 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.063663 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.063700 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:34:23.064258 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.064260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.064278 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.064379 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.065087 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.066042 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.067481 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.067896 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.068044 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.071230 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.071239 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.071239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.071250 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.073156 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.073899 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.073903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.073903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.074012 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.074251 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.074252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.074253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.074655 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.074792 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.075022 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.075173 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
I20260809 16:34:23.075686 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.063367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:39343 (local address 127.0.0.1:42932)
0809 16:34:23.063584 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.063635 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:23.064094 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:23.064486 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:23.064489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:23.064762 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:23.064933 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.064937 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.066165 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.066167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.067204 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.067207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.068026 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.068028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.068100 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.068879 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:23.068891 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:23.071120 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:23.073539 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.073549 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.073552 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.073799 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.074094 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.074095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.074096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.074186 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.074768 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:23.074775 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:23.074853 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 16:34:23.074976 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 16:34:23.075325 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":104,"thread_start_us":125,"threads_started":1}
I20260809 16:34:23.077056 3338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:34:23.077507 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.077385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:43603 (local address 127.0.0.1:34158)
0809 16:34:23.077435 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.077484 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:43603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 16:34:23.077656 3259 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.23.189:43603: connect: Connection refused (error 111)
I20260809 16:34:23.078240 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.078137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:43603 (local address 127.0.0.1:34168)
0809 16:34:23.078187 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.078218 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:43603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 16:34:23.078372 3338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.189:43603: Network error: Client connection negotiation failed: client connection to 127.30.23.189:43603: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.189.keytab.
Entry for principal kudu/127.30.23.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.189.keytab.
I20260809 16:34:23.098599 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.189:43603
--webserver_interface=127.30.23.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.189.keytab
--principal=kudu/127.30.23.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:39343,127.30.23.189:43603,127.30.23.188:42445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:34:23.127667 3338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } attempt: 1
I20260809 16:34:23.128296 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.128155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:43603 (local address 127.0.0.1:34176)
0809 16:34:23.128224 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.128265 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:43603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 16:34:23.128696 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.128595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:43603 (local address 127.0.0.1:34184)
0809 16:34:23.128644 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.128669 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:43603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 16:34:23.128818 3338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.189:43603: Network error: Client connection negotiation failed: client connection to 127.30.23.189:43603: connect: Connection refused (error 111)
W20260809 16:34:23.187629 3347 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:34:23.187865 3347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:23.187884 3347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:23.187906 3347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:23.189019 3347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:23.189055 3347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:23.189069 3347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:23.189082 3347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:23.189095 3347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:23.190759 3347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:39343,127.30.23.189:43603,127.30.23.188:42445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.189.keytab
--principal=kudu/127.30.23.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.189:43603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.23.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:23.191047 3347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:23.191311 3347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:23.196827 3358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:23.196830 3355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:23.196997 3338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } attempt: 2
I20260809 16:34:23.197278 3347 server_base.cc:1034] running on GCE node
W20260809 16:34:23.197434 3356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:23.197507 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.197303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:43603 (local address 127.0.0.1:34192)
0809 16:34:23.197411 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.197455 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:43603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 16:34:23.197824 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.197734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:43603 (local address 127.0.0.1:34208)
0809 16:34:23.197780 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.197807 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.189:43603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 16:34:23.197789 3347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260809 16:34:23.197942 3338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.189:43603: Network error: Client connection negotiation failed: client connection to 127.30.23.189:43603: connect: Connection refused (error 111)
I20260809 16:34:23.198218 3347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:23.199394 3347 hybrid_clock.cc:648] HybridClock initialized: now 1786293263199383 us; error 44 us; skew 500 ppm
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:23.202219 3347 init.cc:377] Logged in from keytab as kudu/127.30.23.189@KRBTEST.COM (short username kudu)
I20260809 16:34:23.202716 3347 webserver.cc:460] Webserver started at http://127.30.23.189:37615/ using document root <none> and password file <none>
I20260809 16:34:23.202953 3347 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:23.203068 3347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:23.203217 3347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:23.204327 3347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/data/instance:
uuid: "dbe2cc585ec646a1ad26eec8241c99dc"
format_stamp: "Formatted at 2026-08-09 16:34:23 on dist-test-slave-j2vl"
server_key: "2c1c5e1b4248ff8aa2d9867427184e8d"
server_key_iv: "4c446fe7f61f0075765d19b06f1b45e8"
server_key_version: "encryptionkey@0"
I20260809 16:34:23.204715 3347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/wal/instance:
uuid: "dbe2cc585ec646a1ad26eec8241c99dc"
format_stamp: "Formatted at 2026-08-09 16:34:23 on dist-test-slave-j2vl"
server_key: "2c1c5e1b4248ff8aa2d9867427184e8d"
server_key_iv: "4c446fe7f61f0075765d19b06f1b45e8"
server_key_version: "encryptionkey@0"
I20260809 16:34:23.206235 3347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:23.207094 3367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:23.207369 3347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:23.207502 3347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/wal
uuid: "dbe2cc585ec646a1ad26eec8241c99dc"
format_stamp: "Formatted at 2026-08-09 16:34:23 on dist-test-slave-j2vl"
server_key: "2c1c5e1b4248ff8aa2d9867427184e8d"
server_key_iv: "4c446fe7f61f0075765d19b06f1b45e8"
server_key_version: "encryptionkey@0"
I20260809 16:34:23.207587 3347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:23.216048 3347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:23.218230 3347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:23.218405 3347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:23.222885 3347 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.189:43603
I20260809 16:34:23.222942 3457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.189:43603 every 8 connection(s)
I20260809 16:34:23.223356 3347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/data/info.pb
I20260809 16:34:23.224466 3458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:23.225035 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 3347
I20260809 16:34:23.225152 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-1/wal/instance
I20260809 16:34:23.225389 30814 external_mini_cluster.cc:1428] Setting key 063674316862d5a088f3ac5e0d3264a7
I20260809 16:34:23.226907 3458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.189@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.23.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.23.188@KRBTEST.COM" created.
I20260809 16:34:23.238070 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.227813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.30.23.190:39343)
0809 16:34:23.227884 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.227886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.228101 (+ 215us) server_negotiation.cc:407] Connection header received
0809 16:34:23.228576 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.228578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.228594 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.228672 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.229290 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.230132 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.231181 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.231562 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.231647 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.234346 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.234348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.234348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.234356 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.235976 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.236478 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.236479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.236480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.236583 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.236957 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.236958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.236959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.237092 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.237156 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.237795 (+ 639us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.237939 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:34:23.238215 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.227739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:39343 (local address 127.0.0.1:42936)
0809 16:34:23.227965 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.228010 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:23.228476 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:23.228734 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:23.228737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:23.228992 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:23.229189 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.229193 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.230269 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.230271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.231083 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.231085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.231689 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.231690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.231828 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.232454 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:23.232464 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:23.234202 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:23.236101 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.236111 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.236113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.236386 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.236709 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.236710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.236710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.236869 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.237160 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:23.237166 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:23.237599 (+ 433us) client_negotiation.cc:769] Sending connection context
0809 16:34:23.237754 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 16:34:23.237928 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":119,"threads_started":1}
I20260809 16:34:23.238806 3458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.189@KRBTEST.COM for kudu/127.30.23.189@KRBTEST.COM
I20260809 16:34:23.247020 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.239114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:43603 (local address 127.0.0.1:34212)
0809 16:34:23.239165 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.239176 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:23.239218 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:23.239519 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:23.239520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:23.239644 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:23.239728 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.239730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.240644 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.240645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.241521 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.241524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.242047 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.242048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.242138 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.242775 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:23.242780 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:23.244038 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:23.245936 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.245937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.245938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.246106 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.246345 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.246345 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.246346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.246405 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.246692 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:23.246692 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:23.246731 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 16:34:23.246793 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:34:23.246905 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:34:23.247054 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.239173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.30.23.189:43603)
0809 16:34:23.239349 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.239354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.239378 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:34:23.239402 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.239405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.239422 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.239504 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.239819 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.240562 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.241613 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.241960 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.242025 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.244137 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.244148 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.244149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.244161 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.245828 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.246204 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.246207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.246208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.246265 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.246466 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.246467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.246467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.246619 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.246717 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.246851 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.246959 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260809 16:34:23.247822 3458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:34:23.248176 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.248072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:42445 (local address 127.0.0.1:56632)
0809 16:34:23.248105 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.248153 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260809 16:34:23.248332 3368 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
I20260809 16:34:23.248739 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.248660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:42445 (local address 127.0.0.1:56648)
0809 16:34:23.248690 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.248714 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 16:34:23.248888 3458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:42445: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.23.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.188.keytab.
Entry for principal kudu/127.30.23.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.188.keytab.
I20260809 16:34:23.257089 30814 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskunTR4f/build/release/bin/kudu
/tmp/dist-test-taskunTR4f/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.188:42445
--webserver_interface=127.30.23.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.188.keytab
--principal=kudu/127.30.23.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.23.148:36011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.23.190:39343,127.30.23.189:43603,127.30.23.188:42445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:34:23.289112 3338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } attempt: 3
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for kudu/127.30.23.189@KRBTEST.COM
I20260809 16:34:23.296833 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.289654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34226 (local address 127.30.23.189:43603)
0809 16:34:23.289725 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.289727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.289738 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:34:23.289807 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.289808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.289819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.289893 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.290251 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.290885 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.291612 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.291849 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.291931 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.294170 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.294171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.294172 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.294182 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.295596 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.296012 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.296014 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.296014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.296077 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.296287 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.296288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.296289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.296408 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.296484 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.296648 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.296734 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:34:23.296839 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.289569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:43603 (local address 127.0.0.1:34226)
0809 16:34:23.289644 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.289655 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:23.289727 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:23.289900 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:23.289901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:23.290088 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:23.290163 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.290165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.290977 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.290978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.291520 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.291522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.291927 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.291928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.292011 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.292528 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:23.292533 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:23.294070 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:23.295728 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.295730 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.295730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.295933 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.296151 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.296152 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.296153 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.296218 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.296486 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:23.296487 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:23.296536 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 16:34:23.296622 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:34:23.296731 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 16:34:23.297758 3338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:34:23.298170 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.298049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:42445 (local address 127.0.0.1:56660)
0809 16:34:23.298107 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.298145 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 16:34:23.298151 3458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } attempt: 1
I20260809 16:34:23.298794 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.298657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:42445 (local address 127.0.0.1:56680)
0809 16:34:23.298723 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.298761 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 16:34:23.298898 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.298421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:42445 (local address 127.0.0.1:56670)
0809 16:34:23.298598 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.298635 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":116,"threads_started":1}
W20260809 16:34:23.299013 3338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:42445: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
I20260809 16:34:23.299219 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.299134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:42445 (local address 127.0.0.1:56690)
0809 16:34:23.299174 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.299199 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 16:34:23.299331 3458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:42445: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
I20260809 16:34:23.330189 3338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } attempt: 1
I20260809 16:34:23.330783 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.330637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:42445 (local address 127.0.0.1:56706)
0809 16:34:23.330713 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.330755 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 16:34:23.331193 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.331102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:42445 (local address 127.0.0.1:56716)
0809 16:34:23.331150 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.331176 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 16:34:23.331343 3338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:42445: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
W20260809 16:34:23.346196 3472 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:34:23.346452 3472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:23.346482 3472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:23.346529 3472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:23.347733 3472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:23.347810 3472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:23.347834 3472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:23.347864 3472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:23.347904 3472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:23.349570 3472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.23.148:36011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.23.190:39343,127.30.23.189:43603,127.30.23.188:42445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/krb5kdc/kudu_127.30.23.188.keytab
--principal=kudu/127.30.23.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.23.188:42445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.23.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:23.349884 3472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:23.350153 3472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:34:23.355302 3472 server_base.cc:1034] running on GCE node
W20260809 16:34:23.355244 3482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:23.355348 3485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:23.355247 3481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:23.356009 3472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:23.356370 3472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:23.357522 3472 hybrid_clock.cc:648] HybridClock initialized: now 1786293263357509 us; error 34 us; skew 500 ppm
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:23.360390 3472 init.cc:377] Logged in from keytab as kudu/127.30.23.188@KRBTEST.COM (short username kudu)
I20260809 16:34:23.360913 3472 webserver.cc:460] Webserver started at http://127.30.23.188:38017/ using document root <none> and password file <none>
I20260809 16:34:23.361193 3472 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:23.361243 3472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:23.361436 3472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:23.362562 3472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/data/instance:
uuid: "833d2c20557f4bf2906588a116a97a6b"
format_stamp: "Formatted at 2026-08-09 16:34:23 on dist-test-slave-j2vl"
server_key: "7d16e6046e0096678c8e0a3c51fe31b7"
server_key_iv: "444779cd73633b3179fe81403822d5ed"
server_key_version: "encryptionkey@0"
I20260809 16:34:23.362954 3472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/wal/instance:
uuid: "833d2c20557f4bf2906588a116a97a6b"
format_stamp: "Formatted at 2026-08-09 16:34:23 on dist-test-slave-j2vl"
server_key: "7d16e6046e0096678c8e0a3c51fe31b7"
server_key_iv: "444779cd73633b3179fe81403822d5ed"
server_key_version: "encryptionkey@0"
I20260809 16:34:23.364391 3472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:34:23.365269 3503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:23.365506 3472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:34:23.365607 3472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/data,/tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/wal
uuid: "833d2c20557f4bf2906588a116a97a6b"
format_stamp: "Formatted at 2026-08-09 16:34:23 on dist-test-slave-j2vl"
server_key: "7d16e6046e0096678c8e0a3c51fe31b7"
server_key_iv: "444779cd73633b3179fe81403822d5ed"
server_key_version: "encryptionkey@0"
I20260809 16:34:23.365692 3472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:23.367466 3458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } attempt: 2
I20260809 16:34:23.367916 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.367782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:42445 (local address 127.0.0.1:56730)
0809 16:34:23.367846 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.367887 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 16:34:23.368384 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.368255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:42445 (local address 127.0.0.1:56744)
0809 16:34:23.368322 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.368355 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 16:34:23.368520 3458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.23.188:42445: Network error: Client connection negotiation failed: client connection to 127.30.23.188:42445: connect: Connection refused (error 111)
I20260809 16:34:23.376292 3472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:23.378749 3472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:23.378921 3472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:23.383378 3472 rpc_server.cc:307] RPC server started. Bound to: 127.30.23.188:42445
I20260809 16:34:23.383411 3583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.23.188:42445 every 8 connection(s)
I20260809 16:34:23.383867 3472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/data/info.pb
I20260809 16:34:23.384902 3585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:23.386605 3585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.188@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
I20260809 16:34:23.393970 30814 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskunTR4f/build/release/bin/kudu as pid 3472
I20260809 16:34:23.394106 30814 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskunTR4f/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786292963539133-30814-0/minicluster-data/master-2/wal/instance
I20260809 16:34:23.394369 30814 external_mini_cluster.cc:1428] Setting key 573ccc2e442abc4da6a420167bd41b9d
I20260809 16:34:23.396828 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.387413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42944 (local address 127.30.23.190:39343)
0809 16:34:23.387510 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.387513 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.387691 (+ 178us) server_negotiation.cc:407] Connection header received
0809 16:34:23.388233 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.388234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.388246 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.388316 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.388980 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.389526 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.390637 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.390914 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.391023 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.393482 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.393484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.393484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.393492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.395432 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.395866 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.395868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.395868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.395938 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.396206 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.396207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.396207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.396315 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.396369 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.396691 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.396761 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 16:34:23.397230 3587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.387354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.190:39343 (local address 127.0.0.1:42944)
0809 16:34:23.387566 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.387605 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:23.388115 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:23.388389 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:23.388393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:23.388652 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:23.388875 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.388878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.389652 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.389653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.390537 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.390539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.391099 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.391101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.391272 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.391828 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:23.391837 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:23.393331 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:23.395526 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.395532 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.395534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.395782 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.396007 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.396008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.396008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.396138 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.396379 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:23.396385 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:23.396544 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 16:34:23.396658 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 16:34:23.396965 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":125,"thread_start_us":126,"threads_started":1}
I20260809 16:34:23.397840 3585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.188@KRBTEST.COM
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.188@KRBTEST.COM for kudu/127.30.23.189@KRBTEST.COM
I20260809 16:34:23.404734 3589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.394804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56746 (local address 127.30.23.188:42445)
0809 16:34:23.394990 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.394994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.395035 (+ 41us) server_negotiation.cc:407] Connection header received
0809 16:34:23.395107 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.395109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.395121 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.395187 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.395932 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.397017 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.398034 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.398596 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.398668 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.401320 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.401330 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.401330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.401337 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.403397 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.403821 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.403824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.403824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.403880 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.404125 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.404126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.404127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.404278 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.404440 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.404523 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.404606 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260809 16:34:23.404940 3587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.398133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.189:43603 (local address 127.0.0.1:34230)
0809 16:34:23.398191 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.398204 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:23.398243 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:23.398376 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:23.398377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:23.398528 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:23.398580 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.398582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.399498 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.399498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.399977 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.399978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.400420 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.400421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.400501 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.400929 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:23.400933 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:23.402278 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:23.403863 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.403864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.403865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.404038 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.404252 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.404253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.404253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.404310 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.404574 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:23.404575 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:23.404619 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 16:34:23.404744 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 16:34:23.404843 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 16:34:23.405104 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.398216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34230 (local address 127.30.23.189:43603)
0809 16:34:23.398268 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.398270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.398277 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:34:23.398299 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.398300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.398308 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.398353 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.398697 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.399409 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.400063 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.400343 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.400404 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.402378 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.402379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.402380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.402388 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.403804 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.404116 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.404117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.404118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.404175 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.404385 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.404386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.404386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.404501 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.404572 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.404934 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.405020 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:34:23.405540 3585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:34:23.406517 3338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } attempt: 2
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.188@KRBTEST.COM for kudu/127.30.23.188@KRBTEST.COM
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.190@KRBTEST.COM
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.190@KRBTEST.COM for kudu/127.30.23.188@KRBTEST.COM
I20260809 16:34:23.412979 3587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.405790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:42445 (local address 127.0.0.1:56760)
0809 16:34:23.405821 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.405831 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:23.405866 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:23.405992 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:23.405993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:23.406102 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:23.406131 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.406132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.406919 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.406920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.407651 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.407653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.407994 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.407994 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.408041 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.408473 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:23.408478 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:23.409946 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:23.411881 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.411882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.411882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.412049 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.412246 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.412247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.412247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.412303 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.412613 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:23.412613 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:23.412651 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 16:34:23.412746 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 16:34:23.412858 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260809 16:34:23.413232 3589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.405824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56760 (local address 127.30.23.188:42445)
0809 16:34:23.405855 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.405857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.405864 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:34:23.405919 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.405920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.405930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.405991 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.406200 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.406821 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.407723 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.407921 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.407997 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.410052 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.410053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.410054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.410060 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.411779 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.412134 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.412135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.412135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.412194 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.412391 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.412392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.412392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.412525 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.412590 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.412770 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.412907 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":16}
I20260809 16:34:23.415297 3585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b: Bootstrap starting.
I20260809 16:34:23.415400 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.405677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42960 (local address 127.30.23.190:39343)
0809 16:34:23.405717 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.405719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.405727 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:34:23.405750 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.405751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.405762 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.405825 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.406455 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.407123 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.408130 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.408431 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.408472 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.412088 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.412090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.412090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.412097 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.414163 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.414523 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.414524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.414524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.414582 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.414807 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.414808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.414808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.414947 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.415124 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.415196 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.415280 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:34:23.416023 3585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:23.416126 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.407323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:42445 (local address 127.0.0.1:56766)
0809 16:34:23.407368 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.407377 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:23.407426 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:23.407685 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:23.407687 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:23.407892 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:23.407963 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.407965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.408874 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.408875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.409754 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.409757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.410188 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.410189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.410260 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.410682 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:23.410687 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:23.412493 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:23.414856 (+ 2363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.414857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.414858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.415071 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.415296 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.415296 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.415297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.415350 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.415785 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:23.415786 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:23.415832 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:34:23.415925 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 16:34:23.416030 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 16:34:23.416304 3590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.407394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56766 (local address 127.30.23.188:42445)
0809 16:34:23.407552 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.407554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.407561 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:34:23.407579 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.407580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.407588 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.407651 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.408064 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.408770 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.409868 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.410126 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.410185 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.412984 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.412985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.412985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.412992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.414740 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.415154 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.415155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.415156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.415217 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.415579 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.415579 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.415579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.415700 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.415779 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.415952 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.416031 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260809 16:34:23.416369 3585 log.cc:824] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:23.417347 3585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b: No bootstrap required, opened a new log
I20260809 16:34:23.417829 3338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f: Bootstrap starting.
I20260809 16:34:23.418560 3338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:23.418869 3338 log.cc:824] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:23.419279 3585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } }
I20260809 16:34:23.419440 3585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 833d2c20557f4bf2906588a116a97a6b, State: Initialized, Role: FOLLOWER
I20260809 16:34:23.419525 3585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } }
I20260809 16:34:23.419684 3338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f: No bootstrap required, opened a new log
I20260809 16:34:23.419845 3585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:23.419833 3596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } } }
I20260809 16:34:23.420058 3596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.23.189@KRBTEST.COM
W20260809 16:34:23.422160 3613 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 16:34:23.422812 3338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } }
I20260809 16:34:23.422974 3338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02cfcb8b64384d9f9476907c6813378f, State: Initialized, Role: FOLLOWER
W20260809 16:34:23.422946 3613 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:34:23.423059 3338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } }
I20260809 16:34:23.423318 3617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } } }
I20260809 16:34:23.423472 3617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:34:23.423385 3338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:23.423266 3472 master_runner.cc:186] Error getting master registration for 127.30.23.190:39343: OK, instance_id { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" instance_seqno: 1786293263056908 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:34:23.425848 3472 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 16:34:23.426182 3638 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:34:23.426484 3638 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:34:23.426785 3232 master_runner.cc:186] Error getting master registration for 127.30.23.189:43603: OK, instance_id { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" instance_seqno: 1786293263221631 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:34:23.429111 3232 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:34:23.430047 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.417302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34236 (local address 127.30.23.189:43603)
0809 16:34:23.417371 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.417373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.417382 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:34:23.417415 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.417415 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.417426 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.417488 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.418086 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.418816 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.419784 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.420089 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.420137 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.423217 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.423218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.423219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.423227 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.424994 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.426651 (+ 1657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.426652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.426652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.426717 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.429431 (+ 2714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.429433 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.429433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.429605 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.429792 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.429847 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.429947 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:34:23.431979 30814 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:23.432978 3291 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "02cfcb8b64384d9f9476907c6813378f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42960
I20260809 16:34:23.433123 3291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 16:34:23.433173 3291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:23.434381 3291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } }
I20260809 16:34:23.434670 3291 leader_election.cc:288] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [CANDIDATE]: Term 1 election: Requested vote from peers dbe2cc585ec646a1ad26eec8241c99dc (127.30.23.189:43603), 833d2c20557f4bf2906588a116a97a6b (127.30.23.188:42445)
I20260809 16:34:23.435081 3541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "02cfcb8b64384d9f9476907c6813378f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "833d2c20557f4bf2906588a116a97a6b"
I20260809 16:34:23.435211 3541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:23.436265 3541 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02cfcb8b64384d9f9476907c6813378f in term 1.
I20260809 16:34:23.436537 3252 leader_election.cc:302] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02cfcb8b64384d9f9476907c6813378f, 833d2c20557f4bf2906588a116a97a6b; no voters:
I20260809 16:34:23.436676 3617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:23.436826 3617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [term 1 LEADER]: Becoming Leader. State: Replica: 02cfcb8b64384d9f9476907c6813378f, State: Running, Role: LEADER
I20260809 16:34:23.436929 3617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } }
I20260809 16:34:23.437426 3617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02cfcb8b64384d9f9476907c6813378f. Latest consensus state: current_term: 1 leader_uuid: "02cfcb8b64384d9f9476907c6813378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } } }
I20260809 16:34:23.437538 3617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:23.448838 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.436600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42964 (local address 127.30.23.190:39343)
0809 16:34:23.436667 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.436669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.437913 (+ 1244us) server_negotiation.cc:407] Connection header received
0809 16:34:23.438077 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.438078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.438102 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.438134 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.440982 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.441752 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.442805 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.443107 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.443150 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.445308 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.445309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.445310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.445319 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.447240 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.447679 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.447681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.447681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.447739 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.447936 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.447936 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.447937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.448065 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.448413 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.448648 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.448734 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:34:23.453503 3650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:23.453775 3650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:23.454548 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.437368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34244 (local address 127.30.23.189:43603)
0809 16:34:23.437421 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.437423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.439149 (+ 1726us) server_negotiation.cc:407] Connection header received
0809 16:34:23.439251 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.439252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.439264 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.439295 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.440263 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.441029 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.446169 (+ 5140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.446510 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.446558 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.451156 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.451158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.451159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.451167 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.452746 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.453326 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.453327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.453328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.453432 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.453773 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.453774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.453775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.453935 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.454019 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.454370 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.454463 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:34:23.454618 3590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.438724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56774 (local address 127.30.23.188:42445)
0809 16:34:23.438775 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.438777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.439316 (+ 539us) server_negotiation.cc:407] Connection header received
0809 16:34:23.439629 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.439630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.439641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.439672 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.442386 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.443157 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.447542 (+ 4385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.447881 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.447926 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.451036 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.451038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.451039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.451047 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.453146 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.453594 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.453595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.453595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.453654 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.453985 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.453986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.453986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.454129 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.454225 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.454428 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.454523 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:34:23.456519 3541 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b [term 1 FOLLOWER]: Refusing update from remote peer 02cfcb8b64384d9f9476907c6813378f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:23.456907 3646 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [LEADER]: Connected to new peer: Peer: permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 16:34:23.459661 3458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } attempt: 3
I20260809 16:34:23.460038 3596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02cfcb8b64384d9f9476907c6813378f. Latest consensus state: current_term: 1 leader_uuid: "02cfcb8b64384d9f9476907c6813378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } } }
I20260809 16:34:23.460120 3596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 16:34:23 dist-test-slave-j2vl krb5kdc[3213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293263, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.23.189@KRBTEST.COM for kudu/127.30.23.188@KRBTEST.COM
I20260809 16:34:23.470295 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.460046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.23.188:42445 (local address 127.0.0.1:56780)
0809 16:34:23.460127 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:23.460137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:23.460173 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:23.460531 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:23.460533 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:23.460714 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:23.460768 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.460769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.462393 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.462395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.463209 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:23.463211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.463784 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.463785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:23.463861 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.464442 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:23.464448 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:23.466169 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:23.468759 (+ 2590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.468760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.468761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.469049 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.469320 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:23.469321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:23.469321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:23.469447 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:23.469952 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:23.469954 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:23.470018 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 16:34:23.470089 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 16:34:23.470199 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 16:34:23.470983 3617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02cfcb8b64384d9f9476907c6813378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } } }
I20260809 16:34:23.471104 3617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:23.471268 3590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:23.460308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56780 (local address 127.30.23.188:42445)
0809 16:34:23.460365 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.460368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.460378 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:34:23.460399 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.460400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.460410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.460472 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.460868 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.462271 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.463325 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:23.463640 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:23.463700 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:23.466285 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:23.466287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:23.466287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:23.466296 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:23.468603 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.469117 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.469118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.469119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.469229 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:23.469531 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:23.469532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:23.469532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:23.469808 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:23.469915 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:23.470869 (+ 954us) server_negotiation.cc:299] Negotiation successful
0809 16:34:23.471009 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:34:23.471660 3596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02cfcb8b64384d9f9476907c6813378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } } }
I20260809 16:34:23.471751 3596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:34:23.472031 3658 mvcc.cc:205] Tried to move back new op lower bound from 7316657207115071488 to 7316657207038332928. Current Snapshot: MvccSnapshot[applied={T|T < 7316657207115071488}]
I20260809 16:34:23.473053 3458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc: Bootstrap starting.
I20260809 16:34:23.473647 3650 catalog_manager.cc:1347] Generated new cluster ID: 828a100cd57040569b094cc5c2c606c9
I20260809 16:34:23.473703 3650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:23.473817 3458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:23.474108 3458 log.cc:824] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:23.475765 3458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc: No bootstrap required, opened a new log
I20260809 16:34:23.477911 3458 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } }
I20260809 16:34:23.478048 3458 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe2cc585ec646a1ad26eec8241c99dc, State: Initialized, Role: FOLLOWER
I20260809 16:34:23.478122 3458 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } }
I20260809 16:34:23.478528 3409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "02cfcb8b64384d9f9476907c6813378f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dbe2cc585ec646a1ad26eec8241c99dc"
I20260809 16:34:23.478638 3409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:23.479467 3409 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02cfcb8b64384d9f9476907c6813378f in term 1.
I20260809 16:34:23.479715 3408 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc [term 1 FOLLOWER]: Refusing update from remote peer 02cfcb8b64384d9f9476907c6813378f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:23.478461 3458 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:23.480350 3665 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:23.478444 3670 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } } }
I20260809 16:34:23.480981 3670 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:34:23.484858 3670 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02cfcb8b64384d9f9476907c6813378f. Latest consensus state: current_term: 1 leader_uuid: "02cfcb8b64384d9f9476907c6813378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } } }
I20260809 16:34:23.484965 3670 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:34:23.485005 3670 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02cfcb8b64384d9f9476907c6813378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } } }
I20260809 16:34:23.485042 3670 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:34:23.487212 3686 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:34:23.487347 3686 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:34:23.487851 3684 mvcc.cc:205] Tried to move back new op lower bound from 7316657207115071488 to 7316657207038332928. Current Snapshot: MvccSnapshot[applied={T|T < 7316657207115071488}]
I20260809 16:34:23.488770 3665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02cfcb8b64384d9f9476907c6813378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02cfcb8b64384d9f9476907c6813378f" member_type: VOTER last_known_addr { host: "127.30.23.190" port: 39343 } } peers { permanent_uuid: "dbe2cc585ec646a1ad26eec8241c99dc" member_type: VOTER last_known_addr { host: "127.30.23.189" port: 43603 } } peers { permanent_uuid: "833d2c20557f4bf2906588a116a97a6b" member_type: VOTER last_known_addr { host: "127.30.23.188" port: 42445 } } }
I20260809 16:34:23.488869 3665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:23.489271 3650 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:23.490299 3650 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:23.497713 3650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02cfcb8b64384d9f9476907c6813378f: Generated new TSK 0
I20260809 16:34:23.497932 3650 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:34:23.528399 3590 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:23.526591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56788 (local address 127.30.23.188:42445)
0809 16:34:23.526640 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.526642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.527650 (+ 1008us) server_negotiation.cc:407] Connection header received
0809 16:34:23.527757 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.527758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.527771 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.527802 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.528310 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56788: BlockingRecv error: recv got EOF from 127.0.0.1:56788 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 16:34:23.529451 3341 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:23.527053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.30.23.190:39343)
0809 16:34:23.527096 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.527098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.528631 (+ 1533us) server_negotiation.cc:407] Connection header received
0809 16:34:23.528726 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.528727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.528738 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.528767 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.529380 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42970: BlockingRecv error: recv got EOF from 127.0.0.1:42970 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 16:34:23.530265 3464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:23.527345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34252 (local address 127.30.23.189:43603)
0809 16:34:23.527399 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.527401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.529704 (+ 2303us) server_negotiation.cc:407] Connection header received
0809 16:34:23.529809 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.529810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.529822 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.529852 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.530205 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34252: BlockingRecv error: recv got EOF from 127.0.0.1:34252 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 16:34:23.532843 3464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:23.532046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34256 (local address 127.30.23.189:43603)
0809 16:34:23.532085 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.532087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.532282 (+ 195us) server_negotiation.cc:407] Connection header received
0809 16:34:23.532411 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.532412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.532431 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.532461 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.532794 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34256: BlockingRecv error: recv got EOF from 127.0.0.1:34256 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 16:34:23.536083 3464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:23.535250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34272 (local address 127.30.23.189:43603)
0809 16:34:23.535286 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.535288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.535546 (+ 258us) server_negotiation.cc:407] Connection header received
0809 16:34:23.535656 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.535657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.535668 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.535698 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.536037 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34272: BlockingRecv error: recv got EOF from 127.0.0.1:34272 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:34:23.540387 3464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:23.539432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34278 (local address 127.30.23.189:43603)
0809 16:34:23.539475 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.539477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.539791 (+ 314us) server_negotiation.cc:407] Connection header received
0809 16:34:23.539904 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.539905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.539918 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.539950 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.540334 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34278: BlockingRecv error: recv got EOF from 127.0.0.1:34278 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 16:34:23.546659 3464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:23.545756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34290 (local address 127.30.23.189:43603)
0809 16:34:23.545797 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.545799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.546055 (+ 256us) server_negotiation.cc:407] Connection header received
0809 16:34:23.546167 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.546168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.546182 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.546213 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.546602 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34290: BlockingRecv error: recv got EOF from 127.0.0.1:34290 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:34:23.557482 3464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:23.556188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34300 (local address 127.30.23.189:43603)
0809 16:34:23.556239 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.556242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.556600 (+ 358us) server_negotiation.cc:407] Connection header received
0809 16:34:23.556733 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.556734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.556749 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.556787 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.557406 (+ 619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34300: BlockingRecv error: recv got EOF from 127.0.0.1:34300 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 16:34:23.576568 3464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:23.575521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34304 (local address 127.30.23.189:43603)
0809 16:34:23.575572 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.575575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.575923 (+ 348us) server_negotiation.cc:407] Connection header received
0809 16:34:23.576035 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.576036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.576048 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.576079 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.576501 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34304: BlockingRecv error: recv got EOF from 127.0.0.1:34304 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 16:34:23.611096 3464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:23.610067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34308 (local address 127.30.23.189:43603)
0809 16:34:23.610112 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.610115 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.610420 (+ 305us) server_negotiation.cc:407] Connection header received
0809 16:34:23.610525 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.610526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.610539 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.610569 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.611030 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34308: BlockingRecv error: recv got EOF from 127.0.0.1:34308 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:34:23.677390 3464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:23.676470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34322 (local address 127.30.23.189:43603)
0809 16:34:23.676534 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.676536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.676706 (+ 170us) server_negotiation.cc:407] Connection header received
0809 16:34:23.676745 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.676746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.676759 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.676843 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.677297 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34322: BlockingRecv error: recv got EOF from 127.0.0.1:34322 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 16:34:23.807698 3464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:23.806827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34338 (local address 127.30.23.189:43603)
0809 16:34:23.806919 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:23.806922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:23.807021 (+ 99us) server_negotiation.cc:407] Connection header received
0809 16:34:23.807122 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:23.807124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:23.807137 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:23.807214 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:23.807629 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34338: BlockingRecv error: recv got EOF from 127.0.0.1:34338 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 16:34:24.065913 3464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:24.065228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34352 (local address 127.30.23.189:43603)
0809 16:34:24.065311 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:24.065313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:24.065322 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:34:24.065376 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:24.065377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:24.065386 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:24.065449 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:24.065845 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34352: BlockingRecv error: recv got EOF from 127.0.0.1:34352 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:34:24.423597 3613 catalog_manager.cc:1259] Loaded cluster ID: 828a100cd57040569b094cc5c2c606c9
I20260809 16:34:24.423687 3613 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b: loading cluster ID for follower catalog manager: success
I20260809 16:34:24.424654 3613 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b: acquiring CA information for follower catalog manager: success
I20260809 16:34:24.424985 3613 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 833d2c20557f4bf2906588a116a97a6b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:34:24.487918 3686 catalog_manager.cc:1259] Loaded cluster ID: 828a100cd57040569b094cc5c2c606c9
I20260809 16:34:24.488050 3686 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc: loading cluster ID for follower catalog manager: success
I20260809 16:34:24.488938 3686 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc: acquiring CA information for follower catalog manager: success
I20260809 16:34:24.489286 3686 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dbe2cc585ec646a1ad26eec8241c99dc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:34:24.582243 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:24.579295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.30.23.189:43603)
0809 16:34:24.579448 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:24.579450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:24.579534 (+ 84us) server_negotiation.cc:407] Connection header received
0809 16:34:24.579574 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:24.579575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:24.579633 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:24.579730 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:24.579880 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:24.580543 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:24.581426 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:24.581626 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:24.581692 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:24.581830 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:24.581958 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:24.582022 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:24.582108 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 16:34:24.582128 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":103,"threads_started":1}
I20260809 16:34:24.586865 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:24.584188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56790 (local address 127.30.23.188:42445)
0809 16:34:24.584311 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:24.584313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:24.584328 (+ 15us) server_negotiation.cc:407] Connection header received
0809 16:34:24.584367 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:24.584368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:24.584408 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:24.584484 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:24.584640 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:24.585136 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:24.585989 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:24.586203 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:24.586271 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:24.586455 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:24.586571 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:24.586645 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:24.586751 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 16:34:24.586772 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
I20260809 16:34:24.591634 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:24.588816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56800 (local address 127.30.23.188:42445)
0809 16:34:24.588859 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:24.588861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:24.588889 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:34:24.588983 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:24.588985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:24.588996 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:24.589068 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:24.589225 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:24.589873 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:24.590799 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:24.591104 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:24.591173 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:24.591337 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:24.591374 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:24.591447 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:24.591510 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 16:34:24.591533 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:34:24.591953 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:24.588950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.30.23.189:43603)
0809 16:34:24.589008 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:24.589010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:24.589162 (+ 152us) server_negotiation.cc:407] Connection header received
0809 16:34:24.589282 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:24.589283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:24.589297 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:24.589367 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:24.589535 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:24.590273 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:24.591147 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:24.591415 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:24.591473 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:24.591655 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:24.591691 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:24.591722 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:24.591857 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 16:34:24.591886 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:34:24.592607 3807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:24.588735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42986 (local address 127.30.23.190:39343)
0809 16:34:24.588839 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:24.588842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:24.589153 (+ 311us) server_negotiation.cc:407] Connection header received
0809 16:34:24.589223 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:24.589224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:24.589284 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:24.589466 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:24.589602 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:24.590343 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:24.591295 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:24.591607 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:24.591683 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:24.591916 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:24.592035 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:24.592118 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:24.592344 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 16:34:24.592371 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":50,"threads_started":1}
I20260809 16:34:24.594961 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 3232
I20260809 16:34:24.611147 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 3347
I20260809 16:34:24.627462 30814 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskunTR4f/build/release/bin/kudu with pid 3472
2026-08-09T16:34:24Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3174 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6425 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (301099 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST